Arvados

e.g., when a workflow step fails to start and exceeds MaxDisptachAttempts, a-c-r logs that like this:

2025-04-07T16:17:03.088631091Z INFO [container mark-duplicates_2] pirca-xvhdp-bu1zt795wlyxsr2 state is Committed
2025-04-07T16:18:38.905393926Z INFO [workflow bwamem-gatk-report_2] starting step mark-duplicates_2
2025-04-07T16:18:38.905530411Z INFO [step mark-duplicates_2] start
2025-04-07T16:26:02.964049781Z INFO [step mark-duplicates_2] completed success
2025-04-07T18:02:02.572908811Z INFO [container mark-duplicates_2] pirca-xvhdp-bu1zt795wlyxsr2 is Final
2025-04-07T18:02:02.647185846Z ERROR [step mark-duplicates] Output is missing expected field keep:723ccb12f21518b2fe936ec68d2d5927+2900/helper/bwamem-gatk-report-wf.cwl#mark-duplicates/dupbam
2025-04-07T18:02:02.729833977Z ERROR [step mark-duplicates] Output is missing expected field keep:723ccb12f21518b2fe936ec68d2d5927+2900/helper/bwamem-gatk-report-wf.cwl#mark-duplicates/dupmetrics
2025-04-07T18:02:02.861320386Z WARNING [step mark-duplicates] completed permanentFail

Problems with this:

• The "Output is missing expected field" error is the primary one that gets reported in Workbench. This sounds like user error (a bug in the workflow or malformed inputs) but it's not, it's a system error. a-c-r should report that explicitly to help the user accurately diagnose what happened.
• The "completed success" log throws you further off the trail because it sounds like the process is okay but it isn't. I am guessing this log means something like "a dispatcher locked the container" or something. Whatever it actually means, it should be clarified to make more sense in context.