Arvados

h1. Container secret mounts

This is a proposed modification to [[Containers API]].

Add a "secret_mounts" serialized field to containers and container requests.

"secret_mounts" has the same form and behavior as "mounts", except:

* Only literal content is allowed (kind=text or kind=json)

* Current value is never returned in a container request or container API response

* Current value can be retrieved from a new API (@/arvados/v1/containers/$uuid/secret_mounts@) which must be authenticated by the container's own runtime token

* Never appears in container logs

* Never appears in the Arvados logs table

* Never appears in websocket updates

* Never appears in API server request logs

It is an error for the same key (mount path) to appear in both mounts and secret_mounts. It should not be possible to _commit_ a container request with this error condition, although it might be possible to _save._

A secret_mounts key (path) cannot be

* equal to the container's output path,

* a descendant of the container's output path, or

* an ancestor of the container's output path (currently, this is a moot point because secret mounts are not directories)

(PA) I would modify this restriction.  Secret file mounts should be allowed to appear under the output directory, but must be _excluded_ when capturing output.  The reasoning is that this aligns better with the CWL InitialWorkDir feature which pre-populates the output directory.  Pre-populating other directories is technically legal but less portable (only works if you are using containers, which is an ok assumption for Arvados but not for CWL generally).

For clarity, some ways in which secret_mounts behaves like mounts are:

* Non-identical secret_mounts disqualifies a container for reuse. The mere existence of secret_mounts does not disqualify.

* secret_mounts can be set via container_requests#create and container_requests#update APIs

* secret_mounts cannot be null, but can be an empty hash

* keys of secret_mounts are paths in the container's filesystem, and always begin with "/"

h3. Secret environment

Add a "secret_environment" serialized field to containers and container requests.

"secret_environment" has the same form and behavior as "environment", except:

* Current value is never returned in a container request or container API response

* Current value can be retrieved from a new API (@/arvados/v1/containers/$uuid/secret_environment@) which must be authenticated by the container's own runtime token

* Never appears in container logs

* Never appears in the Arvados logs table

* Never appears in websocket updates

* Never appears in API server request logs

It is an error for the same key (environment var) to appear in both environment and secret_environment. It should not be possible to _commit_ a container request with this error condition, although it might be possible to _save._

For clarity, some ways in which secret_environment behaves like environment are:

* Non-identical secret_environment disqualifies a container for reuse. The mere existence of secret_environment does not disqualify.

* secret_environment can be set via container_requests#create and container_requests#update APIs

* secret_environment cannot be null, but can be an empty hash

* keys of secret_environment are names of environment variables

h3. Secret command

Add a "secret_environment" serialized field to containers and container requests.

"secret_command" has the same form and behavior as "command", except:

* Current value is never returned in a container request or container API response

* Current value can be retrieved from a new API (@/arvados/v1/containers/$uuid/secret_command@) which must be authenticated by the container's own runtime token

* Never appears in container logs

* Never appears in the Arvados logs table

* Never appears in websocket updates

* Never appears in API server request logs

To run, "secret_command" is merged with "command".  If secret_command is empty, it is ignored, otherwise they must be arrays of the same size.  Entries in "command" which are "null" take their value from the corresponding entry in "secret_command".  It is an error for the same entry in command and secret_command to to both have a value or both be null.

For clarity, some ways in which secret_command behaves like command are:

* Non-identical secret_command disqualifies a container for reuse. The mere existence of secret_command does not disqualify.

* secret_command can be set via container_requests#create and container_requests#update APIs

* secret_command cannot be null, but can be an empty list