-
Notifications
You must be signed in to change notification settings - Fork 625
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
NonBlockingFileIO.read(...) don't error if more than 4 GiB is requested #2591
Comments
We should also upgrade some of the |
We don't need > 32 bit chunks. Reasonable chunk sizes are in kiB or MiB |
Agree but I'm talking about the |
Fair enough! |
A
ByteBuffer
can never hold more than 4 GiB of data. But theNonBlockingFileIO.read(...)
functions don't complain if more than that is requested. They should error saying that this request can't be fulfilled.Likewise the
readChunked
functions cap out at (IIRC) 2 GiB and don't complain if you request more than 2 GiB chunks.Both need to error.
So
read
: Work through 4 GiB, error > 4 GiBreadChunked
: Work up through 2 GiB, error > 2 GiB (the reason for the 2 GiB limit is that IIRC the underlying posix functions don't work if >CInt.max
, this should be checked though).The text was updated successfully, but these errors were encountered: