Bug #15814
openRunning a workflow from WB2 exposes secret inputs
0%
Description
In the container.json log of e51c5-xvhdp-ctxc9vlsbq7ae3x you can see:
"api_key_per_sample": { "$include": "/secrets/s0" }
In the container.json log of e51c5-xvhdp-mkzv6lz6mnphv7b, you can see the api_key_per_sample in plaintext.
For reference, the way secrets are handled in arvados-cwl-runner:
- the submitter process takes the secret string and adds a "text" type mount at /secrets/s0 (s1, s2, etc) to the container request
- In the input object, the parameter is replaced with "$include": "/secrets/s0"
- The workflow runner process (inside the container) loads the input object and processes the $input directive, which reads /secrets/s0 and replaces it with the contents of the file
- The workflow runner internally swaps the secret for a placeholder to avoid printing it in logging (including debug logging)
- The command line tool uses InitialWorkDir to define the credential files
- It observes that the file contains the placeholder for the secret
- The file is moved to secret_mounts and the placeholder is replaced by the real secret
- secret_mounts are hidden from all API responses except when crunch-run requests the "self" container. Secrets are wiped from the database when the container is finished
Implementation¶
The part that workbench 2 needs to handle is:
- Recognizing which inputs are secrets (requires looking for cwltool:Secrets in the workflow's hints or requirements sections).
- Obscuring the secret with a "password" type text box
- When constructing the container request, moving secrets into the "secret_mounts" part, and replacing them in the input object with the $include reference.
Updated by Tom Morris over 5 years ago
- Target version changed from 2020-01-02 Sprint to 2020-01-15 Sprint
Updated by Peter Amstutz over 5 years ago
The broader problem is that specifying secrets requires some extra work when creating the container request, and this code is essentially duplicated between arvados-cwl-runner and workbench. As a result, a-c-r knows how to do it, and workbench(1|2) doesn't. This almost certainly isn't the only example of inconsistent behavior between them.
I recommend exploring the option of submitting the workflow using a high level API, specifically the GA4GH Workflow Execution Service (WES). The implementation of the high level API can invoke arvados-cwl-runner which already has the logic to construct the low level container request. Submitted for consideration: #15918 #15917
Updated by Peter Amstutz over 5 years ago
- Target version changed from 2020-01-15 Sprint to Arvados Future Sprints
Updated by Peter Amstutz about 5 years ago
- Target version deleted (
Arvados Future Sprints) - Release set to 20
Updated by Peter Amstutz about 3 years ago
- Target version set to 2022-02-16 sprint
- Release changed from 20 to 46
Updated by Peter Amstutz about 3 years ago
- Subject changed from Running a workflow from WB1 & WB2 exposes secret inputs to Running a workflow from WB2 exposes secret inputs
Updated by Peter Amstutz about 3 years ago
- Target version changed from 2022-02-16 sprint to 2022-03-02 sprint
Updated by Peter Amstutz about 3 years ago
- Target version changed from 2022-03-02 sprint to 2022-03-16 sprint
Updated by Peter Amstutz about 3 years ago
- Assigned To changed from Peter Amstutz to Daniel Kutyła
Updated by Peter Amstutz about 3 years ago
- Target version changed from 2022-03-16 sprint to 2022-04-13 Sprint
Updated by Peter Amstutz about 3 years ago
- Target version changed from 2022-04-13 Sprint to 2022-04-27 Sprint
Updated by Peter Amstutz almost 3 years ago
- Target version changed from 2022-04-27 Sprint to 2022-05-11 sprint
Updated by Peter Amstutz almost 3 years ago
- Target version changed from 2022-05-11 sprint to 2022-05-25 sprint
Updated by Peter Amstutz almost 3 years ago
- Target version changed from 2022-05-25 sprint to 2022-06-08 sprint
Updated by Peter Amstutz almost 3 years ago
- Target version changed from 2022-06-08 sprint to 2022-06-22 Sprint
Updated by Peter Amstutz almost 3 years ago
- Target version changed from 2022-06-22 Sprint to 2022-07-06
Updated by Peter Amstutz almost 3 years ago
- Target version changed from 2022-07-06 to 2022-07-20
Updated by Peter Amstutz almost 3 years ago
- Target version changed from 2022-07-20 to 2022-08-03 Sprint
Updated by Peter Amstutz almost 3 years ago
- Target version changed from 2022-08-03 Sprint to 2022-08-17 sprint
Updated by Peter Amstutz almost 3 years ago
- Target version changed from 2022-08-17 sprint to 2022-08-31 sprint
Updated by Peter Amstutz over 2 years ago
- Target version changed from 2022-08-31 sprint to 2022-09-28 sprint
Updated by Peter Amstutz over 2 years ago
- Target version changed from 2022-09-28 sprint to 2022-10-12 sprint
Updated by Peter Amstutz over 2 years ago
- Target version changed from 2022-10-12 sprint to 2022-10-26 sprint
Updated by Peter Amstutz over 2 years ago
- Target version changed from 2022-10-26 sprint to 2022-11-09 sprint
Updated by Peter Amstutz over 2 years ago
- Target version changed from 2022-11-09 sprint to 2022-11-23 sprint
Updated by Peter Amstutz over 2 years ago
- Target version changed from 2022-11-23 sprint to 2022-12-21 Sprint
Updated by Peter Amstutz over 2 years ago
- Target version changed from 2022-12-21 Sprint to 2023-01-18 sprint
Updated by Peter Amstutz over 2 years ago
- Target version changed from 2023-01-18 sprint to 2023-02-01 sprint
Updated by Peter Amstutz over 2 years ago
- Target version changed from 2023-02-01 sprint to 2023-02-15 sprint
Updated by Peter Amstutz about 2 years ago
- Target version changed from 2023-02-15 sprint to To be groomed