Describe the bug
ByteArrayDecoderPlain::read can read fewer values than requested if its input buffer is exhausted early on an exact record boundary. The relevant (edited for brevity) code is
let mut read = 0;
let buf = self.buf.as_ref();
while self.offset < self.buf.len() && read != to_read {
if self.offset + 4 > buf.len() {
return Err(ParquetError::EOF("eof decoding byte array".into()));
}
...
read += 1;
}
self.max_remaining_values -= to_read;
Ok(to_read)
Within the loop a check is made for a premature EOF, but if the buffer ends on an exact boundary, the loop will terminate, but we still decrement by and return to_read rather than read.
To Reproduce
A unit test which fails on main:
#[test]
fn test_plain_decoder_reports_values_actually_read() {
// The page claims to contain two values, but its buffer contains only
// one complete PLAIN-encoded BYTE_ARRAY value.
let buffer = Bytes::from_static(&[3, 0, 0, 0, b'f', b'o', b'o']);
let mut decoder = ByteArrayDecoderPlain::new(buffer, 2, Some(2), false);
let mut output = OffsetBuffer::<i32>::with_capacity(2);
assert_eq!(decoder.read(&mut output, 2).unwrap(), 1);
assert_eq!(output.values.as_slice(), b"foo");
assert_eq!(output.offsets.as_slice(), &[0, 3]);
assert_eq!(decoder.max_remaining_values, 1);
}
Expected behavior
read should either report the actual number of values read, or return the same EOF error in this scenario.
Additional context
This was discovered by Codex during a review of #10420.
Describe the bug
ByteArrayDecoderPlain::readcan read fewer values than requested if its input buffer is exhausted early on an exact record boundary. The relevant (edited for brevity) code isWithin the loop a check is made for a premature EOF, but if the buffer ends on an exact boundary, the loop will terminate, but we still decrement by and return
to_readrather thanread.To Reproduce
A unit test which fails on main:
Expected behavior
readshould either report the actual number of values read, or return the same EOF error in this scenario.Additional context
This was discovered by Codex during a review of #10420.