Skip to content

storage: read blob in chunk will get exception when reach the end the blob #723

Closed

Description

@sudison

Following code is used to read blob chunk by chunk:

try (ReadChannel reader = blob.reader()) {
  WritableByteChannel channel = Channels.newChannel(outputStream);
  ByteBuffer bytes = ByteBuffer.allocate(64 * 1024);
  while (reader.read(bytes) > 0) {
    bytes.flip();
    channel.write(bytes);
    bytes.clear();
  }
}

reader.read() will throw exception:

com.google.gcloud.storage.StorageException: java.lang.NullPointerException

at com.google.gcloud.storage.StorageException.translateAndThrow(StorageException.java:74)
at com.google.gcloud.storage.BlobReadChannel.read(BlobReadChannel.java:159)

The root cause is, reader is blindly reading chunks from cloud storage, if it reaches beyond the end of the blob, then cloud storage will throw an error: 416 Requested range not satisfiable.

Activity

  1. mziccard commented on Mar 7, 2016

    @mziccard
    Contributor

    Hey @sudison thanks for the report. Unfortunately I did not manage to reproduce the error. Which version of gcloud-java are you using? How big is the blob you are trying to read?

    Thanks

  2. sudison commented on Mar 7, 2016

    @sudison
    Author

    blob size is 2 x 1024 x 1024, gcloud version is 0.1.4

  3. aozarov commented on Mar 7, 2016

    @aozarov
    Contributor

    @sudison Thanks, I can reproduce that and will fix that together with #720

  4. added a commit that references this issue on Mar 8, 2016
  5. aozarov commented on Mar 8, 2016

    @aozarov
    Contributor

    #725 should fix both issues (waiting for a reviewer before it can get merged).

  6. mziccard commented on Mar 8, 2016

    @mziccard
    Contributor

    @sudison We just merged #725 so this should be fixed in master now. We will release a new version of gcloud-java soon to make this fix available through in maven. I'll write here as soon as we release the new artifact.

  7. sudison commented on Mar 8, 2016

    @sudison
    Author

    Thanks for the quick fixes. Tested locally, it works.

  8. mziccard commented on Mar 8, 2016

    @mziccard
    Contributor

    @sudison Thanks to you for reporting the issues! Version 0.1.5 is out now and includes the fixes!

  9. 14 remaining items

  10. added a commit that references this issue on Dec 22, 2025
  11. added a commit that references this issue on Mar 11, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions

    Sponsor
    SponsoredKunjungi sekarang
    Promo