Support #18894: [document] how Keep manifest size can be predicted/calculated - Arvados

Actions

Copy link

Support #18894

closed

[document] how Keep manifest size can be predicted/calculated

Added by Ward Vandewege almost 3 years ago. Updated almost 3 years ago.

Status:

Resolved

Priority:

Normal

Assigned To:

Peter Amstutz

Category:

Target version:

2022-04-27 Sprint

Start date:

04/15/2022

Due date:

% Done:

100%

Estimated time:

(Total: 0.00 h)

Story points:

Release:

Arvados 2.4.1

Release relationship:

Auto

Description

Peter wrote this up a while ago, it should be added to the documentation at https://doc.arvados.org/architecture/manifest-format.html

=================================================================================================================
Here's a formula for estimating manifest size, assuming efficiently packed blocks. This is including the block signatures, which makes it bigger than the size of the manifest in the database, because that is what is actually transferred to/from the API server and stored in RAM by arv-mount.

sum(size of all directory paths)
+ (total data size / 64 MB) * 94
+ sum(size of all file names)
+ sum(number of files * 20) =================================================================================================================

Subtasks 1 (0 open — 1 closed)