Arvados

Problem¶

As of 3.1.2 a lot of columns stored as JSONB have no validation and RailsAPI will happily store types other than the one intended. When controller gets this and tries to display it, it errors out and returns 500:

$ arv container_request create -o - --select='["uuid"]' <<'EOF'
{
    "name": "new issue output storage classes",
    "output_storage_classes": "default",
    …
}
EOF
{
 "etag":"",
 "kind":"arvados#containerRequest",
 "uuid":"z2a01-xvhdp-vm54ww9ta3h16lb" 
}
$  curl -i -H "Authorization: Bearer $ARVADOS_API_TOKEN" "https://$ARVADOS_API_HOST/arvados/v1/container_requests/z2a01-xvhdp-vm54ww9ta3h16lb" 
HTTP/1.1 500 Internal Server Error
…

{"errors":["json: cannot unmarshal string into Go struct field ContainerRequest.output_storage_classes of type []string"]}
$  curl -i -H "Authorization: Bearer $ARVADOS_API_TOKEN" "https://$ARVADOS_API_HOST/arvados/v1/container_requests" 
HTTP/1.1 500 Internal Server Error
…

{"errors":["json: cannot unmarshal string into Go struct field ContainerRequest.items.output_storage_classes of type []string"]}

Solution¶

If the JSON is valid but cannot be deserialized because it does not conform to the expected structure, controller should return a non-retryable error. In this case the expectation is that RailsAPI is behaving predictably and the bad data isn't going to fix itself.

A note about the status code: 4xx represent a problem with the request, and this isn't that, so ideally we'd use an appropriate 5xx code. However, we have comments that suggest that the Google API client libraries retry all 5xx status codes. For example, in source:sdk/python/arvados/api.py:

        # googleapiclient only retries 403, 429, and 5xx status codes.
        # If we got another 4xx status that we want to retry, convert it into
        # 5xx so googleapiclient handles it the way we want.

As a practical matter a requirement for the status code is that existing client code should not retry it.