Story #3761
Updated by Tom Clegg over 10 years ago
At startup, keepstore will set up:
* A BlockList instance called "currentPullList" (exists but currently called pullmgr)
* A BlockPuller instance that performs work on the current pull list (e.g., @go BlockPuller.New(currentPullList).Run()@)
The resulting BlockPuller goroutine will use the BlockList interface to learn what work is to be done.
The BlockPuller's Run method will perform the following algorithm:
* Get the current pull list by calling getList() on the list given during instantiation.[1]
* Walk the pull list. Attempt each {block,server} entry one time. Log each attempt and each success.
* When a pull succeeds, call removeItem() on the list. (This will delete all pull list entries for that block.)[2]
* Repeat.
"Attempt" means:
* Generate a random API token.[3]
* Generate a permission signature using the random token, timestamp 60 seconds in the future, and desired block hash.
* Using this token & signature, retrieve the given block from the given keepstore server.
* Verify checksum.
* Write to storage, just as if it had been provided by a client in a PUT transaction.
fn1. Optionally/eventually, in cases where the current pull list is empty, GetList() will block until something is available.
fn2. If this isn't already done, start by stubbing removeItem() as a no-op, and revisit after everything else if there's time left in this story.
fn3. Currently, Keep doesn't actually verify API tokens, just the permission signature, so a random token is just as effective as a real one.