Project

General

Profile

Actions

Bug #10629

closed

[FUSE] slow enumerating files by collection uuid

Added by Tom Morris over 8 years ago. Updated about 8 years ago.

Status:
Resolved
Priority:
Normal
Assigned To:
Category:
Performance
Target version:
Start date:
11/24/2016
Due date:
% Done:

100%

Estimated time:
(Total: 0.00 h)
Story points:
1.0

Description

It takes 38 minutes (!!) to enumerate 242K files. During this time the arv-mount process is pegged at 100% CPU and no network traffic is being done. The manifest text totals just 9MB (9243914 char), so it's taking over 4 min / MB to parse this.

$ time find keep/by_id/e51c5-4zz18-l3dq8bw20uwz0qd -print | wc -l
241751
real 38m0.969s
user 0m0.224s
sys 0m0.300s

$ wc *.manifest
2497 252893 9243914 e51c5-4zz18-l3dq8bw20uwz0qd.manifest


Subtasks 1 (0 open1 closed)

Task #11114: Review 10629-fuse-listing-perfResolvedPeter Amstutz11/24/2016

Actions

Related issues 2 (1 open1 closed)

Related to Arvados - Bug #6019: [FUSE] `ls` in a large home directory (4400+ items) takes too longClosed05/13/2015

Actions
Related to Arvados - Bug #9732: [FUSE] performance issues stat() in pythonNew08/04/2016

Actions
Actions

Also available in: Atom PDF