IPIP-550: PBNode field ordering - #550
Open
achingbrain wants to merge 9 commits into
Open
Conversation
🚀 Build Preview on IPFS ready
|
Having `Link` messages before the `Data` message in `PBNode` messages means for HAMT shards you have to read every `Link` before you can process any of them, since the hash type/fanout values used to calculate the prefix each directory entry has is stored in the `Data` field at the end of the message. This IPIP suggests allowing writers to encode the `Data` field first, this allows readers to use streaming parsers that can skip out on reading links if they have already reached the directory entry they are interested in. Further discussion can be found in #533
achingbrain
force-pushed
the
ipip-550-pbnode-field-ordering
branch
from
August 24, 2026 13:48
37fdb60 to
51f16e3
Compare
lidel
reviewed
Aug 24, 2026
lidel
left a comment
lidel
left a comment
Contributor
There was a problem hiding this comment.
Thanks, left some quick first-pass drive-by comments below.
achingbrain
added a commit
to ipld/ipld
that referenced
this pull request
Aug 25, 2026
Writing the `Links` first makes reading HAMT data more expensive since, if you are only trying to read a certain path, you have to read all of the `Links` before you can process any of them, as the config needed to calculate the hash prefix length is stored in the `Data` field at the end of the message. Swap the fields round so `Data` can be read before `Links`. Further discussion: ipfs/specs#533 IPIP: ipfs/specs#550
lidel
added a commit
to ipfs/gateway-conformance
that referenced
this pull request
Aug 27, 2026
* feat: tests for IPIP-550 PBNode field ordering Gateways must resolve UnixFS content regardless of PBNode field order: the legacy Links-first encoding and the Data-first order proposed for the unixfs-v1-2026 profile. - fixtures: pbnode-field-orders.car, single-entry Directory and HAMTShard in both orders, byte-identical to the IPIP table - tests: path resolution through all four roots, raw block responses byte-exact for all five blocks Refs ipfs/specs#550 * chore: changelog entry under v0.14.1
Make the Data-first ordering opt-in per profile and align the text with DAG-PB codec behavior. - unixfs.md: Profiles registry section with names implementations SHOULD use; both PBNode orderings with reader/writer rules and UnixFS precedence over the legacy DAG-PB codec spec; data-first test vectors in the appendix - ipip-0550: accurate motivation and compatibility, PBNode field order profile parameter (links-first default), two-way security notes, alternatives, fixtures relabeled and linked to tagged gateway-conformance v0.14.1 CARs, working code links
Contributor
|
Pushed 17444ef and b707207: the ordering is now defined per profile. Does this look sensible @achingbrain? unixfs.md gains a Profiles registry ( Working code in Go:
That makes ratification relatively low-risk: whoever needs the streaming-friendly order can lock it in via the profile, and everyone else, legacy DAGs and libraries included, keeps preexisting behavior. |
…-ordering # Conflicts: # src/unixfs.md
Member
Author
|
Thanks for the updates, all looks good to me. |
1 task
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Having
Linkmessages before theDatamessage inPBNodemessages means for HAMT shards you have to read everyLinkbefore you can process any of them, since the hash type/fanout values used to calculate the prefix each directory entry has is stored in theDatafield at the end of the message.This IPIP suggests allowing writers to encode the
Datafield first, this allows readers to use streaming parsers that can skip out on reading links if they have already reached the directory entry they are interested in.Further discussion can be found in #533