Repository navigation
feature: search cbor for an element at a given (JSON pointer like) path #736
Description
Is your feature request related to a problem?
In the use case I'm working on, data is stored in a single CBOR structure.
Users provide input data via a REST API (as JSON) which is stored at conventional location(s) in the global structure.
Server processes then need to retrieve (and potentially modify) these data and/or create new one, also added in the structure.
This results in semi-large to large data.
The data flow currently consists in:
- decoding the global structure into a generic
map[string]interface{} - retrieving the sub-tree in the generic structure that corresponds to the specifics of the process via a JSON-pointer-like parameter
- decode the sub-tree into the actually needed go structs via
mapstructure(or other means, including re-encoding into json followed by json unmarshaling)
Describe the solution you'd like
It would be more efficient to be able to locate the sub-tree in cbor and decode into structs from there
Solution would be a function Search(keys []string) (offset int, err error) allowing to decode the element at the returned offset.
Describe alternatives you've considered
see description
Additional context
n/a
Activity
elv-gilles commentedon Jan 9, 2026
I implemented this in a branch that is ready here.
Please let me know if you are interested in a PR.
fxamacker commentedon Jan 12, 2026
Hi @elv-gilles, thanks for opening this issue and providing a link to your implementation!
This feature seems useful.
I like your approach of having a separate
Searcherthat avoids modifying the decoder, and using syntax similar to JSON pointer.However, there are some security considerations. The implementation converts CBOR map keys to string (if key is not a CBOR text string), and such conversions can create duplicate map keys.
For example, if a CBOR map has two different keys 123 and "123", then converting integer 123 to string "123" would create a duplicate map key.
Some quick thoughts for consideration:
Searcherprobably need to support all map related decoding options (e.g.DecOptions.DupMapKey) so CBOR map rejected in current approach is also rejected in the new approach.keys []anyinstead ofkeys []string, so map keys are typed and don't need to be converted to string for comparison.JSON Pointer [RFC6901] doesn't distinguish map key vs array index in the path. This can create ambiguity in a CBOR implementation during traversal, especially if we support map key of non-string types without conversion. The same path can be applied to different data structures. For example,
[0, 0]can be a valid path for nested array in an array, nested map in a map (with integer map keys), etc. To resolve this, maybe include a path type (map key or array index) along with the value. For example, instead of[0, 0], we can specify the type to eliminate ambiguity between array index and map key:[{Type: ArrayIndex, Value: 0}, {Type: MapKey, Value: 0}].Thoughts?
elv-gilles commentedon Jan 13, 2026
Hi @fxamacker, thanks for the answer.
I'll look into hardening my implementation with respect to your comments.
On the other hand, as you noticed, the
Searcheris entirely implemented via the decoder.An alternative would consist in exposing a partial view of the decoder API (make it public) and let users of the
cbor/v2library implement features (like search) by themselves. I made a quick trial at this with a ReadSeekDecoder.What's your opinion ?
fxamacker commentedon Jan 18, 2026
Hi @elv-gilles, thanks for the reply!
In most cases, my preference is to separate the high-level API from the low-level API. For example, low-level API can be in its own package, so users can choose the set of API that they are most comfortable with.
I already have an issue open for creating streaming API. Once streaming API is created, I think it can be used for features (like search) under the hood, similar to ReadSeekDecoder.
Before creating the new streaming API (issue #666), I need to wrap up some in-progress refactoring (e.g., issue #709) that were postponed due to more time-sensitive projects that required weekends, holidays, etc.
elv-gilles commentedon Jan 20, 2026
Hi @fxamacker ok thanks, then I'm putting this on hold (and I think on my side this is not urgent).