Feature #23207
closedUser guide page for running workflows on data in S3
Description
This is something a lot of our users do and there are a lot of options around it. Document how to run workflows in the different scenarios.
Updated by Brett Smith 5 months ago
23207-s3-user-guide @ 9b1d417959360f833808fa52c8c13538161cf547 - developer-run-tests-doc-sdk-java-R: #2254
New doc page as described. I tested the entire process of creating and using stored workflows on my local development cluster; it works. Based on this morning's main. Other checklist items are N/A for a pure doc branch.
We have discussed changes to the credentials API that will necessitate updates to this doc but that's also true for other docs we already have. There's enough here that it's worth reviewing and merging as-is and we can update it with those changes IMO.
Updated by Brett Smith 5 months ago
I just found source:doc/user/cwl/cwl-style.html.textile.liquid which covers a lot of the same stuff at the bottom of the page. I'm going to go over it and unify it into the new page, which IMO is more discoverable and also goes into more detail about setting up external credentials.
Updated by Brett Smith 5 months ago
Now at 41c950466e802ed5ce100114ec0a5f3aa490c2c8 which brings some documentation from cwl-style to the new page, then replaces that entire section with a link to the new page. I'm obviously biased but I think this is a net improvement:
- The new page is more discoverable. While I understand a lot of these download features were originally motivated by performance or convenience concerns, I don't think most users would think that a page titled "Guidelines for Writing High-Performance Portable Workflows" talks about how to load data from S3.
- The new page goes into enough detail about all these features that it's worth breaking out into its own page.