We have implemented a multi-threaded storage uploader which uses BlobWriteChannel per thread to upload a chunk of a local file. (Once all chunks are uploaded, compose operation completes the transfer.) Despite keep-alive being enabled, (http) connections are not re-used, requiring a rather expensive TLS handshake on every subsequent (file) upload.
The problem lies in the current implementation of resumable uploads. Each intermediate PUT request (corresponding to a chunk being uploaded), except the last one, will result in 308 response code (see [1]) and consequently HttpResponse.disconnect (see [2]).
Thus, the corresponding connection is closed, effectively requiring a new handshake for each intermediate chunk in the subsequent upload.
[1] https://cloud.google.com/storage/docs/json_api/v1/how-tos/resumable-upload
[2] https://github.com/google/google-http-java-client/blob/release-1.24.1/google-http-client/src/main/java/com/google/api/client/http/HttpRequest.java#L1074
We have implemented a multi-threaded storage uploader which uses
BlobWriteChannelper thread to upload a chunk of a local file. (Once all chunks are uploaded,composeoperation completes the transfer.) Despitekeep-alivebeing enabled, (http) connections are not re-used, requiring a rather expensive TLS handshake on every subsequent (file) upload.The problem lies in the current implementation of resumable uploads. Each intermediate
PUTrequest (corresponding to a chunk being uploaded), except the last one, will result in308response code (see [1]) and consequentlyHttpResponse.disconnect(see [2]).Thus, the corresponding connection is closed, effectively requiring a new handshake for each intermediate chunk in the subsequent upload.
[1] https://cloud.google.com/storage/docs/json_api/v1/how-tos/resumable-upload
[2] https://github.com/google/google-http-java-client/blob/release-1.24.1/google-http-client/src/main/java/com/google/api/client/http/HttpRequest.java#L1074