Actions
Feature #22471
open
[keep-balance] multi-pass mode
Description
- When posting a new trash/pull list, set X-Keep-List-Prefix header, so keepstore knows which entries to clear
- Run a pass for each prefix, then merge resulting statistics to produce full cluster summary
- when starting a multi-pass sweep, clear replication_confirmed_partial
- after each pass (single prefix), set or reduce replication_confirmed_partial
- after all passes (prefixes) are done, copy replication_confirmed_partial to replication_confirmed
Updated by Tom Clegg about 1 year ago
- Blocked by Feature #22469: [keepstore] support multi-pass mode added
Updated by Tom Clegg about 1 year ago
- Blocked by Feature #22470: [railsapi] add replication_confirmed_partial field to collections added
Updated by Tom Clegg about 1 year ago
- Related to Idea #22218: design for horizontal scaling of Keep-balance to handle increasingly large clusters added
Updated by Tom Clegg 7 months ago
- Related to Idea #19923: keep-balance experimental single-chunk mode added
Updated by Ivan Diaz Alvarez 7 months ago
Hi, we would want to check the progress in the chunk keep-balancer, as this is getting more prioritary for us ,for arkau this would mean:
- Being able to run with 150Gb/4096(#prefixes) RAM, potentially as a continually running service instead of a scheduled task on the HPC cluster
- We are looking into using storage classes for arkau with faster storage tiering, and balancing would be an important part of moving blocks between storage classes
Actions