Arvados

h1. Manual testing plan

h2. Manual testing of SDKs that don't have good coverage

h2. Release candidate builds with ~1 ~2

h2. Workbench2

Need to go through this whole testing plan with both an admin and non-admin account.  Admin only operations are indicated.

h3. Login

* Test login using username/password

* Test login using OpenID Connect

* Test login as federated user

** Login to using remote account on centralized federation (LoginCluster)

** Login to using remote account on peer federation

h3. Left side navigation

* Click on each top level icon (home projects, favorites/public favorite, shared, all processes, instance types, shell access, groups, trash) and confirm that the appropriate page loads with no errors

* Check that the left side panel can be resized

* Check that that the toggle side panel button works as expected

* Check that the +NEW button is disabled unless a project is displayed

h3. Home projects top panel

* Project name should match the logged in user

* Check for expected toolbar buttons

** Details

** User account

** API Details

* Check the buttons have the expected behavior

h3. Project view

h4. Data tab

Should show projects, workflows, and collections (in that order)

Clicking on the name rendered in blue text should navigate to the item

Clicking anywhere else but the name should toggle between selected and not selected

* Unless clicking on the checkbox, clicking on the row clears any other selected items

* When a row is selected, the toolbar moves from the top panel to the data table panel

* Clicking on the check box to the left when a different item is selected selects both items

** The toolbar updates to show only the operations that can be applied to both items

* Clicking "View details" should open the right info panel.  Check that it shows details for the currently selected item.

Check that the toolbar operations are sorted and grouped consistently across different types of items.

Check that the toolbar operations are appropriate to the type of item selected.

Expected toolbar when project is selected:

* View details

* Open in new tab

* Copy link to clipboard

* Open with 3rd party client

* API details

* --- 

* Share

* New project

* Edit project

* Move to

* Move to trash

* ---

* Freeze project

* Add to favorites

* Add to public favorites (admin only)

Expected toolbar when workflow is selected:

* View details

* Open in new tab

* Copy link to clipboard

* API details

* ---

* Run workflow

* Delete workflow

Expected toolbar when collection is selected

* View details

* Open in new tab

* Copy link to clipboard

* Open with 3rd party client

* API details

* --- 

* Share

* Edit collection

* Move to

* Make a copy

* Move to trash

* ---

* Add to favorites

* Add to public favorites

Check that all the toolbar operations work as expected.

Check that right-clicking on a row selects the row and then opens the appropriate context menu.

* Check that the operations apply to the item that was clicked on.

* Check that the operations in the right-click context menu match the toolbar.

Check that clicking on each action in the context menu works as expected.

Check that entering text into the search box refreshes the list with search results

Check that clicking on the three bars in the upper right opens a menu to select columns

Check that enabling/disabling data columns works.  Check that all columns are filled in appropriately for each item, or blank ("-") where no such data applies.

Check that clicking on the "Name" column sorts by name.

* Check sort by "Date created"

* Check sort by "Last modified"

* Check sort by "Trash at"

* Check sort by "Delete at"

h4. Workflows tab

h2. Groups -- standalone and peer federation

# Create group

# Log in as non-admin user.

# Log in as a second non-admin user in a private window for testing sharing.

# check that users cannot see one another

# Add user to group

# Check that users can see one another

h2. Groups -- login cluster federation

# Login cluster: create group

# Satellite cluster: Log in as non-admin user.

# Satellite cluster: Log in as a second non-admin user in a private window for testing sharing.

# Satellite cluster: check that users cannot see one another

# Login cluster: add both users to group

# Satellite cluster: Check that users can see one another

# Satellite cluster: create group

# Satellite cluster: add both users to group

# Satellite cluster: Check that both users can share with the group created on the satellite cluster

h2. Collections

# Create a collection & upload a file

# Add a file

# Rename a file

# Remove a file

# Download one of the files

# Make a sharing link to the collection & check usage from private window

# WB2 ONLY: Mark collection as a favorite, check that it shows up in favorites

# Rename collection

# Edit description

# Add property

# Search for collection by property

# Search for collection by name

# Search for collection by filename

# Search for collection by keyword in description

# Trash collection

# Check that collection can be found in the trash

# Untrash collection

h2. Projects

# Create a project

# Rename a project

# Edit description

# Create a collection inside the project

# Move a collection into the project

# Add read-only sharing permission to the project & check access from other user

# Add read-write sharing permission to project  & check access from other user

# Add manage sharing permission to project  & check access from other user

# Make project as favorite, check that it shows up in favorites

# Search for project by name

# Search for project by keyword in description

# Trash project

# Check that project can be found in the trash

# Untrash project

h2. Workflows

# Upload workflow with arvados-cwl-runnner --create-workflow

# Browse workflow

# Select workflow to run

# Choose input file

# Watch it run

## Check logging

## Check live updates

## Check links to input & output

# Check that it shows up on dashboard, All Processes

# Check graph for the container request

# check graph on the input collection ("used by")

# check graph on output collection ("provenance")

h2. Federation

h3. Peer federation

2 or more clusters are configured with a 'Remoteclusters' entry in config.yml.

h3. Login cluster federation

2 or more clusters are configured with a 'Remoteclusters' entry in config.yml. One of the clusters is the 'login cluster', which means the *other* clusters have a section like this in their config (clsr1 is the login cluster):

<pre>

Clusters:

  clsr2:

    Login:

      LoginCluster: clsr1

</pre>

h2. Misc

# As admin, create a "public favorite" and make sure users see it.

# As admin, deactivate a user. Make sure that user can't log back in

# Add a cluster for multi-site search.

# Upload ssh key & check view

# Create git repo & check view

# As admin, add virtual machine access & check view