Arvados

The trash-sweep code in source:services/api/lib/sweep_trashed_objects.rb can land in a state where the database spends a lot of time/memory trying, but no progress is made.

• sweep_if_stale calls sweep_now approximately every trash_sweep_interval seconds, even if the last call to sweep_now is still running. (This should be fixed: only one sweep should run at a time.)
• sweep_now runs queries that can affect an unbounded number of rows because there is no batch size limit. (There should be a configurable batch size.)
• sweep_now runs 5 of these queries each time. It's not clear from the code whether these all share a single transaction, but if they do, it's a big transaction, and the transaction isolation isn't needed. (These statements are independent of one another and should be done with autocommit on to avoid unnecessary transaction overhead.)