Project

General

Profile

Actions

Feature #23207

closed

User guide page for running workflows on data in S3

Added by Brett Smith 5 months ago. Updated 5 months ago.

Status:
Resolved
Priority:
Normal
Assigned To:
Category:
Documentation
Target version:
Story points:
-
Release relationship:
Auto

Description

This is something a lot of our users do and there are a lot of options around it. Document how to run workflows in the different scenarios.


Subtasks 1 (0 open1 closed)

Task #23209: Review 23207-s3-user-guideResolvedBrett Smith10/15/2025Actions
Actions #1

Updated by Brett Smith 5 months ago

23207-s3-user-guide @ 9b1d417959360f833808fa52c8c13538161cf547 - developer-run-tests-doc-sdk-java-R: #2254

New doc page as described. I tested the entire process of creating and using stored workflows on my local development cluster; it works. Based on this morning's main. Other checklist items are N/A for a pure doc branch.

We have discussed changes to the credentials API that will necessitate updates to this doc but that's also true for other docs we already have. There's enough here that it's worth reviewing and merging as-is and we can update it with those changes IMO.

Actions #2

Updated by Brett Smith 5 months ago

  • Subtask #23209 added
Actions #3

Updated by Brett Smith 5 months ago

I just found source:doc/user/cwl/cwl-style.html.textile.liquid which covers a lot of the same stuff at the bottom of the page. I'm going to go over it and unify it into the new page, which IMO is more discoverable and also goes into more detail about setting up external credentials.

Actions #4

Updated by Brett Smith 5 months ago

  • Status changed from New to In Progress
Actions #5

Updated by Brett Smith 5 months ago

Now at 41c950466e802ed5ce100114ec0a5f3aa490c2c8 which brings some documentation from cwl-style to the new page, then replaces that entire section with a link to the new page. I'm obviously biased but I think this is a net improvement:

  • The new page is more discoverable. While I understand a lot of these download features were originally motivated by performance or convenience concerns, I don't think most users would think that a page titled "Guidelines for Writing High-Performance Portable Workflows" talks about how to load data from S3.
  • The new page goes into enough detail about all these features that it's worth breaking out into its own page.

developer-run-tests-doc-sdk-java-R: #2254

Actions #6

Updated by Tom Clegg 5 months ago

23207-s3-user-guide LGTM, thanks

Actions #7

Updated by Brett Smith 5 months ago

  • Status changed from In Progress to Resolved
Actions

Also available in: Atom PDF