[amrit110]

Why

On prod, every fit from one lab failed this morning with a 500. The traceback ended in OSError: [Errno 36] File name too long.

• The disk tier named each context's files with the hex of its dataset_id, which is the tenant plus the caller's table name. Hex doubles the length, so a table name over about 80 characters passes the 255-byte file name limit.
• Every fit first drops any stale spilled copy of its id. That unlink() raised ENAMETOOLONG, which isn't FileNotFoundError, so the fit itself failed, whether or not anything had been spilled.
• Short names kept working, so the status probe stayed green.

What changes

• _file_stem: ids whose hex name would pass the 255-byte limit (hex over 245, ids over 122 bytes) are named sha256-<digest>, a fixed length. The meta file already records the id.
• Atomic writes: payload and meta are written to a short temporary name, then os.replaced, so a crash never leaves a torn file.
• Startup cleanup: a hashed pair whose meta can't be read is removed, since its id can't be recovered and nothing could ever load or evict it.
• Restart indexing: reads the id from a hashed file's metadata, and ignores a hashed file whose id doesn't hash back to its name.
• Compatibility: every id that had a working hex name before (up to 122 bytes) keeps it, so spill files written by an older tabctx still load after an upgrade.

Testing

• 6 new tests in tests/unit/test_spill.py:
  • a long id round-trips with every file name at most 255 bytes;
  • delete of a long id never raises;
  • short ids keep their hex names;
  • a warm restart indexes long ids;
  • the restart index ignores a hashed meta file that doesn't match its id, and one that isn't JSON;
  • the engine fits, spills, restores and refits a long id. This reproduces the prod failure.
• Against the old code: 4 of the 6 fail, including the engine refit.
• Review fixes: 3 more tests: the exact old hex limit is kept (a 122-byte id keeps its hex name and loads after a restart, a 123-byte one is hashed); a torn hashed meta is removed at restart instead of leaking disk past capacity; a failed write leaves no files. All 3 fail on this PR's first version.
• Full unit suite: 1052 passed. pre-commit is clean.

Prod runs 0.17.1, so this needs a release and an image bump in inference-platform to reach it.

[codecov-commenter]

Codecov Report

❌ Patch coverage is 95.00000% with 2 lines in your changes missing coverage. Please review.
✅ Project coverage is 90.64%. Comparing base (ffa1b05) to head (9351931).

Files with missing lines	Patch %	Lines
src/tabctx/cache/spill.py	95.00%	2 Missing ⚠️

❌ Your changes status has failed because you have indirect coverage changes. Learn more about Unexpected Coverage Changes and reasons for indirect coverage changes.

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #129      +/-   ##
==========================================
+ Coverage   90.60%   90.64%   +0.03%     
==========================================
  Files          62       62              
  Lines        5899     5931      +32     
==========================================
+ Hits         5345     5376      +31     
- Misses        554      555       +1     

Flag	Coverage Δ
integration	34.12% <20.00%> (-0.05%)	⬇️
unit	90.45% <95.00%> (+0.03%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines	Coverage Δ
src/tabctx/cache/spill.py	96.15% <95.00%> (+0.82%)	⬆️

... and 2 files with indirect coverage changes

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.