Actions
Bug #9001
closedcrunchstat-summary fails on year old log file
Status:
Resolved
Priority:
Normal
Assigned To:
Peter Amstutz
Category:
-
Target version:
-
Story points:
-
Description
Trying to use it on a job from 2/24/2015. I guess something changed with the way we log things and it hits a ValueError.
~/arvados/tools/crunchstat-summary/bin/crunchstat-summary --job qr1hi-8i9sb-030iriqh13rvfok --format text > qr1hi-8i9sb-030iriqh13rvfok.txt Traceback (most recent call last): File "/home/bcosc/arvados/tools/crunchstat-summary/bin/crunchstat-summary", line 14, in <module> cmd.run() File "/home/bcosc/.local/lib/python2.7/site-packages/crunchstat_summary-0.1.20160412153631-py2.7.egg/crunchstat_summary/command.py", line 56, in run return self.summer.run() File "/home/bcosc/.local/lib/python2.7/site-packages/crunchstat_summary-0.1.20160412153631-py2.7.egg/crunchstat_summary/summarizer.py", line 150, in run stat, line, e)) ValueError: Error parsing run stat in "2015-02-24_16:52:10 qr1hi-8i9sb-030iriqh13rvfok 8083 0 stderr crunchstat: Running [/usr/bin/docker.io run --rm=true --attach=stdout --attach=stderr --attach=stdin -i --user=crunch --cidfile=/tmp/crunch-job/qr1hi-ot0gb-pqig664l3n38kvz.cid --sig-proxy --dns 10.13.66.165 --dns 172.17.42.1 --volume=/tmp/crunch-job/src:/tmp/crunch-job/src:ro --volume=/tmp/crunch-job/opt:/tmp/crunch-job/opt:ro --volume=/tmp/crunch-job/task/compute38.1.keep:/keep:ro --volume=/tmp --env=CRUNCH_JOB_BIN=/usr/local/arvados/src/services/crunch/crunch-job --env=TASK_SEQUENCE=0 --env=TASK_KEEPMOUNT=/keep --env=JOB_PARAMETER_COMMAND=ARRAY(0x2cc7c80) --env=CRUNCH_SRC_COMMIT=83a9390a05bbffc2e4ea95dd693af3ab3547fa12 --env=TASK_QSEQUENCE=0 --env=JOB_PARAMETER_REFERENCE=3514b8e5da0e8d109946bc809b20a78a+5698 --env=CRUNCH_INSTALL=/tmp/crunch-job/opt --env=CRUNCH_REFRESH_TRIGGER=/tmp/crunch_refresh_trigger --env=ARVADOS_API_TOKEN=4c3mmeij7flwquw30x6mqbejvrooyfeiq3ccvmb1mwjad1nzrk --env=CRUNCH_WORK=/tmp/crunch-job/work --env=CRUNCH_TMP=/tmp/crunch-job --env=TASK_TMPDIR=/tmp/crunch-job-task-work/compute38.1 --env=JOB_UUID=qr1hi-8i9sb-030iriqh13rvfok --env=JOB_PARAMETER_TASK.FOREACH=ARRAY(0x2cb33d0) --env=JOB_PARAMETER_B=HASH(0x2a9dbf0) --env=CRUNCH_JOB_UUID=qr1hi-8i9sb-030iriqh13rvfok --env=TASK_SLOT_NUMBER=1 --env=CRUNCH_SRC_URL=/var/lib/arvados/internal.git --env=TASK_SLOT_NODE=compute38 --env=JOB_SCRIPT=run-command --env=CRUNCH_NODE_SLOTS=8 --env=JOB_PARAMETER_SAMPLE_ID=my_sample --env=JOB_PARAMETER_TASK.STDOUT=$(param) --env=JOB_PARAMETER_D=d0136bc494c21f79fc1b6a390561e6cb+2778 --env=JOB_PARAMETER_PARAM=HASH(0x2cc7cb0) --env=TASK_WORK=/tmp/crunch-job-task-work/compute38.1 --env=ARVADOS_API_HOST=qr1hi.arvadosapi.com --env=JOB_WORK=/tmp/crunch-job-work --env=TASK_UUID=qr1hi-ot0gb-pqig664l3n38kvz --env=JOB_PARAMETER_EACHREAD=$(dir $(d)) --env=CRUNCH_SRC=/tmp/crunch-job/src --env=HOME=/tmp/crunch-job-task-work/compute38.1 f4eafaf1e2d738e0f8d947feb725b5945f0219c5c4956eec6e164a0788abbab8 stdbuf --output=0 --error=0 perl - /tmp/crunch-job/src/crunch_scripts/run-command] ": ValueError('could not convert string to float: [/usr/bin/docker.io',)
Updated by Brett Smith almost 10 years ago
Bryan,
Your diagnosis is spot-on: crunchstat-summary doesn't know how to parse too-old logs. How often do you wish it could?
Updated by Bryan Cosca almost 10 years ago
I think not often enough to change the code (it's only happened once)
Updated by Brett Smith almost 10 years ago
- Target version set to Arvados Future Sprints
Updated by Peter Amstutz over 9 years ago
Branch @ 9001-summary-skip-malformed-lines
Updated by Lucas Di Pentima over 9 years ago
Reviewed the fix, looks good to me. Please merge.
Updated by Peter Amstutz over 9 years ago
- Status changed from New to Resolved
- % Done changed from 0 to 100
Applied in changeset arvados|commit:6c0bf267d795a3ca49c3258c9490714c9e18d333.
Updated by Tom Morris about 9 years ago
- Assigned To set to Peter Amstutz
- Target version deleted (
Arvados Future Sprints)
Actions