fix: opt-in dag-pb field order control (IPIP-550) - #11439
Open
lidel wants to merge 5 commits into
Open
Conversation
Opt-in Data-first PBNode field ordering via the new unixfs-v1-2026 config profile, per IPIP-550. Defaults and the preexisting unixfs-v0-2015 and unixfs-v1-2025 profiles are unchanged and keep their CIDs. - config: Import.UnixFSPBNodeFieldOrder (links-first default, data-first) and the unixfs-v1-2026 profile applying unixfs-v1-2025 settings plus data-first - core/node: wires merkledag.DefaultPBNodeFieldOrder from config - deps: boxo bump to the ipfs/boxo#1212 encoder commit - test/cli: byte-exact fixtures from the IPIP-550 table; pinned CIDs for existing profiles unchanged Refs ipfs/specs#550
Temporary pin to the ipfs/gateway-conformance#304 head so the PBNode field ordering tests run against kubo. Switch back to a tagged release once one ships.
|
Review the following changes in direct dependencies. Learn more about Socket for GitHub.
|
v0.14.1 shipped the ipfs/gateway-conformance#304 tests, so the moving v0.14 tag covers them again.
lidel
marked this pull request as ready for review
August 27, 2026 20:04
lidel
marked this pull request as draft
August 30, 2026 11:38
State what the profile actually changes: every dag-pb node with both Data and Links gets a new CID, files larger than one chunk included, and data already in MFS is upgraded to the new order on first read (plain `ipfs files ls` or `stat` included), a sharded root before its child shards; the MFS root is re-encoded by any command that starts a node. - config/profile.go, docs/config.md: profile description and the unixfs-v1-2026 section - docs/changelogs/v0.43.md: highlight leads with the need (readers get the HAMT layout before links) and the upgrade-on-first-use behavior
Pins the ipfs/boxo#1212 branch tip: data-first bytes derive from dagpb.AppendEncode so link sorting is inherited, unknown field order values return an error, and pinned end-to-end CIDs cover the profile.
lidel
marked this pull request as ready for review
August 30, 2026 20:05
This was referenced Aug 30, 2026
Open
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
Streaming a HAMT-sharded directory is more expensive than it needs to be: the parameters needed to interpret its entries live in PBNode
Data, encoded after all the links, so a reader looking for one entry reads every link first. ipfs/specs#550 proposes an opt-inData-first field order behind a newunixfs-v1-2026profile; kubo had no way to select it.Fix
Import.UnixFSPBNodeFieldOrderoption (defaultlinks-first) andunixfs-v1-2026config profile (unixfs-v1-2025plusdata-first), wired to the encoder from fix(unixfs): customizable dag-pb order via unixfs-v1-2026 profile (IPIP-550) boxo#1212test/cliasserts byte-exact IPIP-550 fixtures under the new profile, and that the pinned CIDs of the default,unixfs-v0-2015, andunixfs-v1-2025behaviors are unchangedImportant
Opt-in fix for people who need to produce data in a format that is more efficient to stream.
Nothing changes unless a user applies the new profile; reading both orders was already supported. Opting in changes the CIDs of new directories, HAMT shards, and files larger than one chunk, and upgrades data already in MFS: the first
ipfs filescommand that reads a directory re-encodes it in the new order with a new CID. Depends on ipfs/boxo#1212; the CI pin and the boxo pseudo-version go back to tags once boxo and gateway-conformance releases ship.