Project

General

Profile

Actions

Feature #19088

closed

Expose properties as S3 tags in keep-web

Added by Peter Amstutz over 2 years ago. Updated about 2 years ago.

Status:
Resolved
Priority:
Normal
Assigned To:
Category:
Keep
Target version:
Start date:
06/27/2022
Due date:
% Done:

100%

Estimated time:
(Total: 0.00 h)
Story points:
-
Release relationship:
Auto

Description

Even though S3 doesn't support searching/filtering on tags, applications can still use tags to do client-side filtering.

  • Directory placeholder for projects includes properties
  • Directory placeholder for collection includes properties
  • Every file inside a collection will return that collection's properties
  • Despite AWS having a limit of 10 tags, 128 characters for keys and 256 characters for values, we will send all the tags without limits.
  • Return the raw ids, no vocabulary interpretation
  • Read only
  • JSON values (such as lists) are returned in their JSON text representation

Subtasks 1 (0 open1 closed)

Task #19190: Review 19088-s3-properties-tagsResolvedPeter Amstutz06/27/2022

Actions
Actions #1

Updated by Peter Amstutz over 2 years ago

  • Category set to Keep
Actions #3

Updated by Peter Amstutz over 2 years ago

  • Description updated (diff)
Actions #4

Updated by Peter Amstutz over 2 years ago

  • Target version set to 2022-06-08 sprint
Actions #5

Updated by Peter Amstutz over 2 years ago

  • Target version changed from 2022-06-08 sprint to 2022-05-25 sprint
Actions #6

Updated by Peter Amstutz over 2 years ago

  • Description updated (diff)
Actions #7

Updated by Peter Amstutz over 2 years ago

  • Target version changed from 2022-05-25 sprint to 2022-06-08 sprint
Actions #8

Updated by Peter Amstutz over 2 years ago

  • Target version changed from 2022-06-08 sprint to 2022-06-22 Sprint
Actions #9

Updated by Peter Amstutz over 2 years ago

  • Assigned To set to Tom Clegg
Actions #10

Updated by Peter Amstutz over 2 years ago

  • Target version changed from 2022-06-22 Sprint to 2022-07-06
Actions #11

Updated by Tom Clegg over 2 years ago

  • Status changed from New to In Progress
Actions #13

Updated by Peter Amstutz over 2 years ago

Tom Clegg wrote in #note-12:

19088-s3-properties-tags @ dd8f1b0527995bc5ad47710d3a483fa18b827bc6 -- developer-run-tests: #3202

Is there any particular reason you didn't implement GetBucketTagging and GetObjectTagging?

https://docs.aws.amazon.com/AmazonS3/latest/API/API_GetObjectTagging.html

The difference between these and HeadObject seems to be that they returns an XML document rather than embedding it in the headers. Which sounds annoying, but someone might want it, I don't know.

I did some looking, I didn't realize that there's no listing API that can return objects with tags to even be able to do client side filtering efficiently. Clients will have to get the tags of every object separately, which is going to be pretty terrible. Hopefully the keep-web response in that case (once cached) be will be very fast and efficient.

Actions #14

Updated by Tom Clegg over 2 years ago

Oops, not really. I just happened to know about the metadata headers and assumed that's what was being asked.

Being readonly, I think it should be just as easy to export the same info in other ways too, if it turns out people really want to use the tagging API.

Actions #15

Updated by Peter Amstutz over 2 years ago

Tom Clegg wrote in #note-14:

Oops, not really. I just happened to know about the metadata headers and assumed that's what was being asked.

Being readonly, I think it should be just as easy to export the same info in other ways too, if it turns out people really want to use the tagging API.

Per our conversation, we agreed to merge now and support APIs for tagging later if someone asks.

It just occurred to me that we should be totally clear about what is supported and what isn't in the documentation, so someone doesn't try to use e.g. GetObjectTagging and then wonder why it didn't work.

(I'm writing this after you merged the branch so a quick follow-on documentation tweak commit would be appreciated it)

Actions #16

Updated by Tom Clegg over 2 years ago

  • Status changed from In Progress to Resolved
  • % Done changed from 0 to 100

Applied in changeset arvados-private:commit:arvados|607033c33f2001c194fe8c68d0dc17e4bde849da.

Actions #17

Updated by Peter Amstutz about 2 years ago

  • Release set to 47
Actions

Also available in: Atom PDF