Probe reference resolvability through the backend writer - #867
Open
ehennestad wants to merge 2 commits into
Open
ehennestad wants to merge 2 commits into
ehennestad wants to merge 2 commits into
Conversation
This was referenced Aug 25, 2026
ehennestad
force-pushed
the
zarr-support/6-writer-reference-probe
branch
from
August 25, 2026 17:45
022bb65 to
6b60aba
Compare
ehennestad
marked this pull request as ready for review
August 25, 2026 17:50
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #867 +/- ##
=======================================
Coverage 95.27% 95.27%
=======================================
Files 234 234
Lines 8311 8313 +2
=======================================
+ Hits 7918 7920 +2
Misses 393 393 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
ehennestad
force-pushed
the
zarr-support/6-writer-reference-probe
branch
from
August 26, 2026 17:43
6b60aba to
ee8c44f
Compare
MetaClass.export probed whether ObjectView/RegionView targets exist by calling io.getRefData with writer.FileId, reaching through the abstract writer to a raw HDF5 file id. The probe's return value was discarded; only its success or failure matters, feeding the deferred second export pass in NwbFile.resolveReferences. Give the base Writer a validateReferenceResolvable method that states this contract in backend-neutral terms: return silently when every target resolves, throw 'NWB:getRefData:InvalidPath' when one does not, and let 'NWB:ObjectView:MissingPath' propagate unchanged. HDF5Writer implements it with the same io.getRefData call, so a RegionView probe still verifies the referenced dataset opens. MetaClass.export no longer touches writer.FileId. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The base-writer smoke test compares the class's method set against an explicit list, so the interface method added by this branch must be listed for the suite to pass. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
ehennestad
force-pushed
the
zarr-support/6-writer-reference-probe
branch
from
August 31, 2026 10:42
ee8c44f to
7033673
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Motivation
Background — This PR is part of a series that makes the write path storage-agnostic (stacked on #866), so that a Zarr v3 writer can be added later. Today MatNWB can only write HDF5 files.
Problem — Before writing a typed object,
MetaClass.exportchecks that everyObjectViewandRegionViewproperty points to a target that already exists in the file. If a target does not exist yet, the object is skipped and exported again later byNwbFile.resolveReferences. This check used a raw HDF5 file id (writer.FileId), so it only works with the HDF5 backend. A writer for another storage format cannot provide an HDF5 file id.Solution — Add the check to the writer interface as
validateReferenceResolvable(referenceValue), and call that fromMetaClass.exportinstead of using the file id. The HDF5 implementation does the same thing as before, so nothing changes in how files are written.What changed
validateReferenceResolvable(referenceValue)onio.backend.base.Writer, implemented inio.backend.hdf5.HDF5Writer.MetaClass.exportno longer useswriter.FileId. Behavior is the same: objects with missing reference targets are still exported in the second pass.Implementation notes
The HDF5 implementation calls
io.getRefDatathe same way the export code did before. This keeps the check consistent with the actual reference write: for aRegionView, it verifies that the referenced dataset can be opened, not just that its path exists. The error identifiers thatcaptureReferenceErrorschecks for (NWB:getRefData:InvalidPathandNWB:ObjectView:MissingPath) are unchanged.How to test
Export and read back a file containing an object reference:
Checklist
fix #XXwhereXXis the issue number?🤖 Generated with Claude Code