Arvados

41caa50 on 3145-readonly-volumes

3145-readonly-volumes LGTM

3145-close-early LGTM

We should use pycurl.UPLOAD in KeepClient (refs #5562)

http://curl.haxx.se/libcurl/c/CURLOPT_UPLOAD.html

This sets the method to PUT and sets the "Expect: 100-continue" header.

The "100 Continue" flow means the client is not supposed to send the request body until the server sends a "100 Continue" response. This gives the server a chance to check whether that the request can reasonably be expected to succeed, then either tells the client to continue or sends a final error.

From reading the Go net/http code, Go supports "100 Continue" transparently by automatically sending the "100 Continue" response when the handler starts reading the request body (but not before). So your branch requires no additional changes to support "Expect: 100-continue" (awesome!)

The other benefit of 100 continue is to open the possibility of serializing requests from a given host, so if a host with a lot of cores is trying to send a lot of data from a lot of processes, we can use "100 Continue" with cooperative clients to limit upload to 2-3 simultaneous senders and reduce network stack congestion.

Peter Amstutz wrote:

3145-close-early LGTM

We should use pycurl.UPLOAD in KeepClient (refs #5562)

http://curl.haxx.se/libcurl/c/CURLOPT_UPLOAD.html

This sets the method to PUT and sets the "Expect: 100-continue" header.

• uploading means using the PUT request unless you tell libcurl otherwise.
• Using PUT with HTTP 1.1 implies the use of a "Expect: 100-continue" header.

We're already using PUT with HTTP 1.1, therefore we are already using the Expect header. I haven't tcpdumped this but that's my reading of the docs.

And yes, Go made it really easy to do the server side of this: just don't read the body if you don't want it!

e94da792 lgtm, thanks.

Nice work on the HTTP status codes that are already defined to mean exactly what we want to say! Way better than "504 Gateway Timeout" as a "block too big" response. I looked up the >1 year history of this one and my favorite part was my own change, "3220: Fix HTTP status codes":

-       TooLongError    = &KeepError{504, "Too Long"}
+       TooLongError    = &KeepError{504, "Timeout"}