Repository navigation
Unbounded allocations from untrusted length fields (CWE-770) — fix in PR #203 #204
Description
While auditing embedding-file readers, I found that the binary-format readers in this crate allocate buffers straight from untrusted length fields in the input file. A tiny malicious file can trigger a huge single allocation before any format validation happens.
Measured on 0.18.0 with a counting global allocator:
| Input | Declared field | Largest single allocation |
|---|---|---|
| 20 bytes | Metadata chunk chunk_len = 1 GiB |
1,073,741,824 bytes (before reading any TOML byte) |
| 70 bytes | NdArray chunk rows = 1, cols = 268,435,456 |
1,073,741,824 bytes |
I counted 11 affected allocation sites (Metadata, NdArray, NdNorms, QuantizedArray, vocab strings, word2vec/fastText/floret readers, ...). The full list, measurements, and a fix that grows every buffer from the bytes actually read are in PR #203, together with regression tests.
Anyone parsing embedding files from untrusted sources (user uploads, downloaded models) is exposed to this, since the allocation happens before the parser can reject bad input.
Is this crate still maintained? I'd like to know how users should treat unpatched issues like this one.
Activity