Keep real world performance numbers¶
AWS/S3 numbers¶
Here are some benchmark numbers against AWS S3, as measured in July 2020. They were generated with the keep-exercise tool in an Arvados installation with one dedicated Keepstore node (c5n.2xlarge) and one dedicated node for running keep-exercise (c5n.2xlarge). The Keepstore node was backed by one S3 bucket, in a VPC with an S3 endpoint installed. Versioning, Server Access logging, Static website hosting, Object-level logging and Default encryption were disabled. Object lock, Transfer acceleration and Requester pays were also disabled. There were no Active notifications. Each test consisted of 4 one minute runs, which were averaged into one number. The tests were repeated 3 times, and of those 3 runs, the highest average speed was selected and included in the table below.
The goamz driver is the default Keep S3 driver as of Arvados 2.0. In Arvados 2.1, a new S3 driver is added based on the aws-sdk-go-v2 driver. It offers better read performance but has not had as much production use. To enable the aws-sdk-go-v2 driver, set the UseAWSS3v2Driver flag to true in config.yml, as described on the Arvados documentation: S3 configuration page.
| threads | goamz | aws-sdk-go-v2 | command line |
|---|---|---|---|
| single read | 32.53 MiB/s | 79.48 MiB/s | keep-exercise -repeat 4 -run-time 60s -vary-request -use-index -rthreads 1 -wthreads 1 |
| single write | 39.75 MiB/s | 41.05 MiB/s | keep-exercise -repeat 4 -run-time 60s -vary-request -use-index -rthreads 1 -wthreads 1 |
When using 20 concurrent read or write threads, the numbers from both drivers are:
| threads | goamz | aws-sdk-go-v2 | command line |
|---|---|---|---|
| 20 read | 585.60 MiB/s | 898.93 MiB/s | keep-exercise -repeat 4 -run-time 60s -vary-request -use-index -rthreads 20 -wthreads 0 |
| 20 write | 610.40 MiB/s | 688.25 MiB/s | keep-exercise -repeat 4 -run-time 60s -vary-request -use-index -rthreads 0 -wthreads 20 |
When using 40 concurrent read or write threads, the numbers from both drivers are:
| threads | goamz | aws-sdk-go-v2 | command line | comment |
|---|---|---|---|---|
| 40 read | 1005.30 MiB/s | 1005.35 MiB/s | keep-exercise -repeat 4 -run-time 60s -vary-request -use-index -rthreads 40 -wthreads 0 | |
| 40 write | 653.05 MiB/s | 974.925 MiB/s | keep-exercise -repeat 4 -run-time 60s -vary-request -use-index -rthreads 0 -wthreads 40 | goamz is cpu bound on the keepstore (800%) |
Historical AWS/S3 numbers¶
| Name | Network performance | CPU | RAM | Keepstore | Data Payload Throughput | CPU load | Date tested | Tested by |
| c5.9xlarge | 10 Gigabit | 36 cores | 72 GiB | 0.1.20171212165144.296aa66-1 | Saturates RX and TX (1+ GiB/sec) | ~4 | 2018-01-18 | Ward |
| r4.8xlarge | 10 Gigabit | 32 cores | 244 GiB | 0.1.20171212165144.296aa66-1 | Saturates RX and TX (1+ GiB/sec) | ~4 | 2018-01-17 | Ward |
| r4.large | "up to" 10 Gigabit | 2 cores | 15 GiB | 0.1.20171212165144.296aa66-1 | ~400 MiB/sec but long periods of cap at ~90 MiB/sec | 2018-01-16 | Ward |
Updated by Ward Vandewege over 4 years ago · 11 revisions