Bug #12891
Updated by Tom Clegg over 7 years ago
If a container request is cancelled neither the container's `log` field nor the container requests `log_uuid` field get populated, making it impossible for crunchstat-summary to analyze the portion of the job which did run. e.g. https://workbench.su92l.arvadosapi.com/container_requests/su92l-xvhdp-4j98m0zgu9xst51#Advanced crunchstat-summary should be able to do some analysis on containers that haven't finished yet like jobs in crunch v1 -------- Additional info from #12893: crunch-run tries to save a log file after the container ends, regardless of final state, but (sometimes?) this doesn't work. Example: su92l-xvhdp-4j98m0zgu9xst51 Some possible explanations: * crunch-dispatch-slurm cancels the slurm job as soon as it notices the container is cancelled. crunch-run catches SIGTERM and tries to write the buffered output and logs, but (according to sample logs) seems to give up 30-40 seconds later without actually writing them. * even if crunch-run gets that far, it seems apiserver would refuse to update the output or log field of a container whose state is Cancelled. (It would be helpful to check server logs to determine whether crunch-run is in fact getting as far as calling collections#create and containers#update.)