Project

General

Profile

Actions

Idea #23159

open

Decide and implement a strategy for handling records with bad field data

Added by Brett Smith 6 months ago.

Status:
New
Priority:
Normal
Assigned To:
-
Category:
API
Target version:
-
Start date:
Due date:
Story points:
-
Release relationship:
Auto

Description

At least because of #23152, it is possible for the database to return malformed data for a record, especially for complex fields. We should decide and implement a strategy for handling these records.

The current front-runner idea is to replace the field with a specific value of the correct type that signals "bad data." This is easy enough to implement; it signals the problem directly to the user without requiring admin intervention; it lets clients render as much as possible; and we can prohibit this value in validations to prevent clients from accidentally propagating it.

Other ideas we've batted around and their downsides:

  • We could just return an error. This makes list results very awkward. Failing an entire list request because one item is malformed is generally really rough on overall system health.
  • Instead of replacing one field we could return an entire "malformed record" value. On the plus side, we can basically rig this up so it's impossible for clients to try to do real work with it. On the downside, it's harder for users to correct themselves.

Related issues 2 (1 open1 closed)

Related to Arvados - Bug #23152: Validate schema of complex fieldsResolvedTom CleggActions
Related to Arvados - Bug #23157: Controller should return a non-retryable error if RailsAPI returns valid JSON that cannot be deserializedNewActions
Actions #1

Updated by Brett Smith 6 months ago

  • Related to Bug #23152: Validate schema of complex fields added
Actions #2

Updated by Brett Smith 5 months ago

  • Related to Bug #23157: Controller should return a non-retryable error if RailsAPI returns valid JSON that cannot be deserialized added
Actions

Also available in: Atom PDF