Bug #12748: [crunch2] crunchstat-summary doesn't report elapsed time for crunch2 jobs - Arvados

Custom queries

All assigned issues
All issues assigned for grooming
My issues for grooming
My issues for grooming (no story pts)
Prioritized open issues

Actions

Copy link

Bug #12748

closed

[crunch2] crunchstat-summary doesn't report elapsed time for crunch2 jobs

Added by Tom Morris about 7 years ago. Updated about 6 years ago.

Status:

Resolved

Priority:

Normal

Assigned To:

Peter Amstutz

Category:

Target version:

2018-10-17 sprint

Start date:

10/04/2018

Due date:

% Done:

100%

Estimated time:

(Total: 0.00 h)

Story points:

Release:

Arvados 1.3

Release relationship:

Auto

Description

From examination, the elapsed time calculation is on a crunch1-only code path. This also means that average CPU utilization doesn't get reported since it depends on the elapsed time.

Subtasks 1 (0 open — 1 closed)

Task #12770: Review 12690-12748-crunchstat-summary

Resolved

Tom Clegg

10/04/2018

Actions

History
Notes
Property changes
Associated revisions

Actions

Copy link

Updated by Tom Morris about 7 years ago

Status changed from New to In Progress
Assigned To set to Tom Morris

I've implemented a stop gap using the existing data sources in 42017ce7d, but it may be better to handle this differently for crunch2 by using the first and last timestamps of the crunch-run.txt file

Actions

Copy link

Updated by Tom Morris about 7 years ago

Target version changed from 2017-12-06 Sprint to 2017-12-20 Sprint

Actions

Copy link

Updated by Tom Morris about 7 years ago

Target version changed from 2017-12-20 Sprint to 2018-01-17 Sprint

Actions

Copy link

Updated by Tom Morris about 7 years ago

Branch containing the commit is 12690-container-request-keep-stats.

Problem confirmed in actual crunch2 outputs.

Actions

Copy link

Updated by Tom Morris about 7 years ago

Target version changed from 2018-01-17 Sprint to 2018-01-31 Sprint

Actions

Copy link

Updated by Tom Morris almost 7 years ago

Target version changed from 2018-01-31 Sprint to 2018-02-14 Sprint

Actions

Copy link

Updated by Tom Morris almost 7 years ago

Target version changed from 2018-02-14 Sprint to 2018-02-28 Sprint

Actions

Copy link

Updated by Tom Morris almost 7 years ago

Target version changed from 2018-02-28 Sprint to 2018-03-14 Sprint

Actions

Copy link

Updated by Tom Morris almost 7 years ago

Target version changed from 2018-03-14 Sprint to 2018-03-28 Sprint

Actions

Copy link

#10

Updated by Tom Morris almost 7 years ago

Target version changed from 2018-03-28 Sprint to 2018-04-11 Sprint

Actions

Copy link

#11

Updated by Tom Morris almost 7 years ago

Target version changed from 2018-04-11 Sprint to 2018-04-25 Sprint

Actions

Copy link

#12

Updated by Tom Morris almost 7 years ago

Target version changed from 2018-04-25 Sprint to 2018-05-09 Sprint

Actions

Copy link

#13

Updated by Tom Morris over 6 years ago

Target version changed from 2018-05-09 Sprint to 2018-05-23 Sprint

Actions

Copy link

#14

Updated by Tom Morris over 6 years ago

Target version changed from 2018-05-23 Sprint to 2018-06-06 Sprint

Actions

Copy link

#15

Updated by Tom Morris over 6 years ago

Target version changed from 2018-06-06 Sprint to 2018-06-20 Sprint

Actions

Copy link

#16

Updated by Tom Morris over 6 years ago

Target version changed from 2018-06-20 Sprint to 2018-07-03 Sprint

Actions

Copy link

#17

Updated by Tom Morris over 6 years ago

Target version changed from 2018-07-03 Sprint to 2018-07-18 Sprint

Actions

Copy link

#18

Updated by Tom Morris over 6 years ago

Target version changed from 2018-07-18 Sprint to 2018-08-01 Sprint

Actions

Copy link

#19

Updated by Tom Morris over 6 years ago

Target version changed from 2018-08-01 Sprint to 2018-08-15 Sprint

Actions

Copy link

#20

Updated by Tom Morris over 6 years ago

Target version changed from 2018-08-15 Sprint to 2018-09-05 Sprint

Actions

Copy link

#21

Updated by Peter Amstutz over 6 years ago

The elapsed time is wildly wrong, it should probably be 0 (the actual runtime is less than 10 seconds and doesn't span a full collection interval.

### Summary for extract-metrics (e51c5-dz642-mtnlzmwjjyryis1)
category    metric    task_max    task_max_rate    job_total
blkio:0:0    read    0    0    0
blkio:0:0    write    0    0    0
cpu    cpus    2    -    -
cpu    sys    0    -    0.00
cpu    user    0    -    0.00
cpu    user+sys    0    -    0.00
fuseops    read    0    0    0
fuseops    write    0    0    0
keepcache    hit    0    0    0
keepcache    miss    0    0    0
keepcalls    get    0    0    0
keepcalls    put    0    0    0
mem    cache    0    -    -
mem    pgmajfault    0    -    0
mem    rss    585728    -    -
net:keep0    rx    0    0    0
net:keep0    tx    0    0    0
net:keep0    tx+rx    0    0    0
time    elapsed    86398    -    86398
# Number of tasks: 1
# Max CPU time spent by a single task: 0s
# Max CPU usage in a single interval: 0%
# Overall CPU usage: 0.00%
# Max memory used by a single task: 0.00GB
# Max network traffic in a single task: 0.00GB
# Max network speed in a single interval: 0.00MB/s
# Keep cache miss rate 0.00%
# Keep cache utilization 0.00%
#!! extract-metrics max RSS was 1 MiB -- try runtime_constraints "ram":1020054732

Actions

Copy link

#22