feat(pyramid): support paths in GetDataByIdsWithFlag

docs/docs/en/src/api/index_class.md

@@ -89,6 +89,7 @@ Bit flags for [`GetDataByIdsWithFlag`](#getdatabyidswithflag), combined with bit
 | `DATA_FLAG_EXTRA_INFO` | `0x10` | extra info blobs |
 | `DATA_FLAG_ATTRIBUTE` | `0x20` | attributes |
 | `DATA_FLAG_ID` | `0x40` | ids |
+| `DATA_FLAG_PATH` | `0x80` | Pyramid paths retained with `store_paths` |
 
 ### `WriteFuncType`
 
@@ -244,8 +245,8 @@ See [Graph Enhancement](../advanced/enhance_graph.md).
 | `GetMinAndMaxId` | `tl::expected<std::pair<int64_t, int64_t>, Error> GetMinAndMaxId() const` | Smallest and largest ids in the index. |
 | `GetExtraInfoByIds` | `tl::expected<void, Error> GetExtraInfoByIds(const int64_t* ids, int64_t count, char* extra_infos) const` | Copies extra-info blobs for `ids` into a caller-provided buffer. |
 | `GetRawVectorByIds` | `tl::expected<DatasetPtr, Error> GetRawVectorByIds(const int64_t* ids, int64_t count, Allocator* specified_allocator = nullptr) const` | Returns stored vectors. Values are *close to* the originals but not guaranteed bit-identical (quantization/precision). |
-| `GetDataByIds` | `tl::expected<DatasetPtr, Error> GetDataByIds(const int64_t* ids, int64_t count) const` | Returns all stored data (vectors, attributes, extra info) for `ids`. |
-| `GetDataByIdsWithFlag` | `tl::expected<DatasetPtr, Error> GetDataByIdsWithFlag(const int64_t* ids, int64_t count, uint64_t selected_data_flag) const` | Like `GetDataByIds` but selects fields via [`DATA_FLAG_*`](#data-selection-flags). |
+| `GetDataByIds` | `tl::expected<DatasetPtr, Error> GetDataByIds(const int64_t* ids, int64_t count) const` | Returns the implementation's default stored fields for `ids`; optional fields may require explicit selection. |
+| `GetDataByIdsWithFlag` | `tl::expected<DatasetPtr, Error> GetDataByIdsWithFlag(const int64_t* ids, int64_t count, uint64_t selected_data_flag) const` | Selects supported fields via [`DATA_FLAG_*`](#data-selection-flags). Pyramid paths require `store_paths: true` and `DATA_FLAG_PATH`. |
 | `GetIndexDetailInfos` | `tl::expected<std::vector<IndexDetailInfo>, Error> GetIndexDetailInfos() const` | Lists the introspectable detail fields. See [`IndexDetailInfo`](types.md#index-detail-info). |
 | `GetDetailDataByName` | `tl::expected<DetailDataPtr, Error> GetDetailDataByName(const std::string& name, IndexDetailInfo& info) const` | Fetches one detail-data payload by name. |
 

docs/docs/en/src/indexes/pyramid.md

@@ -103,6 +103,7 @@ Build-time parameters live under `index_param`.
 | `base_io_type` / `precise_io_type` | string | `"block_memory_io"` | Base and reorder storage backends; `uring_io` is available in builds with liburing. |
 | `base_file_path` / `precise_file_path` | string | — | Required for disk-backed storage such as `buffer_io`, `async_io`, `uring_io`, or `mmap_io`. |
 | `store_raw_vector` | bool | `false` | Preserve an FP32 copy for `GetRawVectorByIds` and precise distance-by-id calculations. |
+| `store_paths` | bool | `false` | Top-level switch that preserves the original paths supplied to `Build` and `Add` so `GetDataByIdsWithFlag` can return them when `DATA_FLAG_PATH` is selected. It applies to every configured hierarchy and cannot be overridden per hierarchy. |
 | `index_min_size` | int | `0` | Minimum sub-index size; smaller groups fall back to scan. |
 | `support_duplicate` | bool | `false` | Allow duplicate ids. |
 | `build_thread_count` | int | `1` | Threads used for parallel build. |
@@ -236,6 +237,29 @@ base->NumElements(n)
 index->Build(base);
 ```
 
+### Retrieving paths by ID
+
+Set the top-level build parameter `store_paths` to `true` to retain the original paths for ID-based
+retrieval. Select `DATA_FLAG_PATH` in `GetDataByIdsWithFlag`; the returned paths follow the requested
+ID order. Use `GetPaths()` for the default unnamed hierarchy and `GetPaths(hierarchy_name)` for a
+named hierarchy:
+
+```cpp
+int64_t requested_ids[] = {product_id_b, product_id_a};
+auto data = index->GetDataByIdsWithFlag(
+    requested_ids, 2, DATA_FLAG_ID | DATA_FLAG_PATH).value();
+
+const std::string* site_paths = data->GetPaths("site");
+const std::string* category_paths = data->GetPaths("category");
+```
+
+`GetDataByIds` and `GetDataByIdsWithFlag` calls without `DATA_FLAG_PATH` do not attach path arrays.
+Selecting `DATA_FLAG_PATH` while `store_paths` is `false` returns an invalid-argument error. When
+path storage is enabled, a hierarchy is included only if every requested ID has a recorded path in
+that hierarchy. If even one requested ID was built or added without that hierarchy's path, its
+getter returns `nullptr`; other hierarchies whose requested paths are complete are still returned.
+In single-hierarchy mode, the same completeness rule applies to `GetPaths()`.
+
 ### Searching a specific hierarchy
 
 Specify which hierarchy to search via `"hierarchies"` in the search parameters.
@@ -282,8 +306,11 @@ auto result = index->RangeSearch(
 
 ### Serialize & Deserialize
 
-Multi-hierarchy indexes serialize and deserialize transparently. The serialized
-format includes all hierarchy names and their graph structures:
+Multi-hierarchy indexes serialize and deserialize transparently. The serialized format includes
+all hierarchy names and their graph structures. With `store_paths: true`, both regular and
+streaming serialization also persist the retained original paths, making them available through
+`GetDataByIdsWithFlag` after deserialization. With the default `false`, the graph hierarchy is
+persisted but the original per-ID paths are not:
 
 ```cpp
 // Serialize

docs/docs/en/src/resources/index_parameters.md

@@ -145,11 +145,18 @@ Pyramid build parameters also live under `index_param`:
     "index_param": {
         "base_quantization_type": "sq8",
         "max_degree": 24,
-        "ef_construction": 300
+        "ef_construction": 300,
+        "store_paths": true
     }
 }
 ```
 
+`store_paths` is a top-level Pyramid build parameter and defaults to `false`. Enable it when
+`GetDataByIdsWithFlag` must return the original default or named-hierarchy paths with
+`DATA_FLAG_PATH`; see the
+[Pyramid parameter table](../indexes/pyramid.md#build-parameters) for its completeness and
+persistence semantics.
+
 MRLE with split RaBitQ uses `base_quantization_type: "tq"`,
 `tq_chain: "mrle, rabitq"`, `mrle_dim`, and the
 `rabitq_bits_per_dim_base`/`rabitq_bits_per_dim_precise` pair. It automatically reorders from the

docs/docs/zh/src/api/index_class.md

@@ -87,6 +87,7 @@ struct Index::Checkpoint {
 | `DATA_FLAG_EXTRA_INFO` | `0x10` | extra info 数据块 |
 | `DATA_FLAG_ATTRIBUTE` | `0x20` | 属性 |
 | `DATA_FLAG_ID` | `0x40` | id |
+| `DATA_FLAG_PATH` | `0x80` | Pyramid 通过 `store_paths` 保留的路径 |
 
 ### `WriteFuncType`
 
@@ -239,8 +240,8 @@ RangeSearch(const DatasetPtr& query, float radius, const std::string& parameters
 | `GetMinAndMaxId` | `tl::expected<std::pair<int64_t, int64_t>, Error> GetMinAndMaxId() const` | 索引中最小与最大的 id。 |
 | `GetExtraInfoByIds` | `tl::expected<void, Error> GetExtraInfoByIds(const int64_t* ids, int64_t count, char* extra_infos) const` | 把 `ids` 的 extra-info 数据块拷贝到调用方提供的缓冲区。 |
 | `GetRawVectorByIds` | `tl::expected<DatasetPtr, Error> GetRawVectorByIds(const int64_t* ids, int64_t count, Allocator* specified_allocator = nullptr) const` | 返回已存向量。其值*接近*原始值，但不保证逐位一致（量化/精度）。 |
-| `GetDataByIds` | `tl::expected<DatasetPtr, Error> GetDataByIds(const int64_t* ids, int64_t count) const` | 返回 `ids` 的全部已存数据（向量、属性、extra info）。 |
-| `GetDataByIdsWithFlag` | `tl::expected<DatasetPtr, Error> GetDataByIdsWithFlag(const int64_t* ids, int64_t count, uint64_t selected_data_flag) const` | 类似 `GetDataByIds`，但通过 [`DATA_FLAG_*`](#数据选择标志) 选择字段。 |
+| `GetDataByIds` | `tl::expected<DatasetPtr, Error> GetDataByIds(const int64_t* ids, int64_t count) const` | 返回实现默认提供的已存字段；可选字段可能需要显式选择。 |
+| `GetDataByIdsWithFlag` | `tl::expected<DatasetPtr, Error> GetDataByIdsWithFlag(const int64_t* ids, int64_t count, uint64_t selected_data_flag) const` | 通过 [`DATA_FLAG_*`](#数据选择标志) 选择支持的字段。Pyramid 路径需要同时设置 `store_paths: true` 和 `DATA_FLAG_PATH`。 |
 | `GetIndexDetailInfos` | `tl::expected<std::vector<IndexDetailInfo>, Error> GetIndexDetailInfos() const` | 列出可自省的细节字段。见 [`IndexDetailInfo`](types.md#索引细节信息)。 |
 | `GetDetailDataByName` | `tl::expected<DetailDataPtr, Error> GetDetailDataByName(const std::string& name, IndexDetailInfo& info) const` | 按名称获取一份细节数据负载。 |
 

docs/docs/zh/src/indexes/pyramid.md

@@ -98,6 +98,7 @@ auto result = index->KnnSearch(
 | `base_io_type` / `precise_io_type` | string | `"block_memory_io"` | 底层与精排存储后端；以 liburing 构建时可用 `uring_io` |
 | `base_file_path` / `precise_file_path` | string | — | `buffer_io`、`async_io`、`uring_io`、`mmap_io` 等磁盘存储必须设置 |
 | `store_raw_vector` | bool | `false` | 保留 FP32 原始向量，用于 `GetRawVectorByIds` 和精确的按 ID 距离计算 |
+| `store_paths` | bool | `false` | 顶层开关；保留传给 `Build` 和 `Add` 的原始路径，使 `GetDataByIdsWithFlag` 在选择 `DATA_FLAG_PATH` 时可以返回它们。该开关对所有已配置的 hierarchy 生效，不支持按 hierarchy 覆盖 |
 | `index_min_size` | int | `0` | 子索引的最小规模；小于该值的分区会退化为线性扫描 |
 | `support_duplicate` | bool | `false` | 是否允许重复 ID |
 | `build_thread_count` | int | `1` | 构建阶段并发线程数 |
@@ -226,6 +227,29 @@ base->NumElements(n)
 index->Build(base);
 ```
 
+### 按 ID 取回路径
+
+将顶层构建参数 `store_paths` 设为 `true` 后，Pyramid 会保留原始路径，供按 ID
+取回。在 `GetDataByIdsWithFlag` 中选择 `DATA_FLAG_PATH` 后，返回路径的顺序与请求 ID
+的顺序一致。默认匿名 hierarchy 使用 `GetPaths()`，命名 hierarchy 使用
+`GetPaths(hierarchy_name)`：
+
+```cpp
+int64_t requested_ids[] = {product_id_b, product_id_a};
+auto data = index->GetDataByIdsWithFlag(
+    requested_ids, 2, DATA_FLAG_ID | DATA_FLAG_PATH).value();
+
+const std::string* site_paths = data->GetPaths("site");
+const std::string* category_paths = data->GetPaths("category");
+```
+
+`GetDataByIds` 以及未选择 `DATA_FLAG_PATH` 的 `GetDataByIdsWithFlag` 都不会附带路径
+数组。`store_paths` 为 `false` 时选择 `DATA_FLAG_PATH` 会返回参数错误。开启路径存储
+后，只有当所有请求 ID 在某个 hierarchy 中都有已记录的路径，结果才会包含该
+hierarchy。只要其中一个 ID 在构建或追加时没有提供该 hierarchy 的路径，对应 getter
+就返回 `nullptr`；其他路径完整的 hierarchy 仍会正常返回。单 hierarchy 模式下，
+`GetPaths()` 遵循相同的完整性规则。
+
 ### 检索指定层级
 
 通过检索参数中的 `"hierarchies"` 指定要检索的层级。查询 Dataset 也需要在对应的
@@ -271,7 +295,10 @@ auto result = index->RangeSearch(
 
 ### 序列化与反序列化
 
-多层级索引的序列化和反序列化完全透明。序列化格式包含所有层级名称及其图结构：
+多 hierarchy 索引的序列化和反序列化完全透明。序列化格式包含所有 hierarchy
+名称及其图结构。当 `store_paths: true` 时，常规序列化和 streaming 序列化还会持久化
+已保留的原始路径，因此反序列化后仍可通过 `GetDataByIdsWithFlag` 获取。使用默认值
+`false` 时，图 hierarchy 会被持久化，但不会保留按 ID 索引的原始路径：
 
 ```cpp
 // 序列化

docs/docs/zh/src/resources/index_parameters.md

@@ -138,11 +138,16 @@ Pyramid 构建参数同样放在 `index_param` 下：
     "index_param": {
         "base_quantization_type": "sq8",
         "max_degree": 24,
-        "ef_construction": 300
+        "ef_construction": 300,
+        "store_paths": true
     }
 }
 ```
 
+`store_paths` 是 Pyramid 顶层构建参数，默认值为 `false`。需要通过
+`GetDataByIdsWithFlag` 和 `DATA_FLAG_PATH` 返回默认或命名 hierarchy 的原始路径时应启用它；路径完整性与持久化语义见
+[Pyramid 参数表](../indexes/pyramid.md#构建参数)。
+
 MRLE 与 split RaBitQ 组合使用 `base_quantization_type: "tq"`、
 `tq_chain: "mrle, rabitq"`、`mrle_dim`，以及
 `rabitq_bits_per_dim_base`/`rabitq_bits_per_dim_precise`。该配置会自动从 split base codes

include/vsag/constants.h

@@ -120,6 +120,7 @@ extern const char* const PYRAMID_PARAMETER_HIERARCHY_OP;
 extern const char* const PYRAMID_NO_BUILD_LEVELS;
 extern const char* const PYRAMID_HIERARCHIES;
 extern const char* const PYRAMID_INDEX_MIN_SIZE;
+extern const char* const PYRAMID_STORE_PATHS;
 
 extern const char PART_SLASH;
 extern const char PART_BAR;

include/vsag/index.h

@@ -72,6 +72,7 @@ enum class IndexType {
 #define DATA_FLAG_EXTRA_INFO 0x10
 #define DATA_FLAG_ATTRIBUTE 0x20
 #define DATA_FLAG_ID 0x40
+#define DATA_FLAG_PATH 0x80
 
 using OffsetType = uint64_t;
 using SizeType = uint64_t;
@@ -700,7 +701,7 @@ class Index {
     }
 
     /**
-     * @brief Retrieve all data associated with vectors identified by given IDs.
+     * @brief Retrieve selected data associated with vectors identified by given IDs.
      *
      * This method fetches data stored with the vectors in the index
      * (e.g., attributes, labels, or extra infos).
@@ -709,7 +710,7 @@ class Index {
      * @param count Number of IDs in the 'ids' array.
      * @param selected_data_flag selected data flag, set with DATA_FLAG_*
      * @return tl::expected<DatasetPtr, Error>
-     *         - On success: A DatasetPtr containing the extra data, attribute and vector
+     *         - On success: A DatasetPtr containing the selected supported fields
      *         - On failure: An error object (e.g., invalid ID, out of memory).
      * @note The default base-class implementation returns tl::unexpected(ErrorType::UNSUPPORTED_INDEX_OPERATION) If the index implementation does not support this operation
      *            (default behavior for base class).
@@ -756,19 +757,19 @@ class Index {
     }
 
     /**
-     * @brief Retrieve all data associated with vectors identified by given IDs.
+     * @brief Retrieve the default data fields associated with vectors identified by given IDs.
      *
      * This method fetches data stored with the vectors in the index
      * (e.g., attributes, labels, or extra infos).
      *
      * @param ids Array of vector IDs for which extra information is requested.
      * @param count Number of IDs in the 'ids' array.
      * @return tl::expected<DatasetPtr, Error>
-     *         - On success: A DatasetPtr containing the extra data, attribute and vector
+     *         - On success: A DatasetPtr containing the implementation's default fields
      *         - On failure: An error object (e.g., invalid ID, out of memory).
      * @note The default base-class implementation returns tl::unexpected(ErrorType::UNSUPPORTED_INDEX_OPERATION) If the index implementation does not support this operation
      *            (default behavior for base class).
-     * @note The default implementation returns all data which in current index
+     * @note Optional fields may require explicit selection through GetDataByIdsWithFlag.
      */
     [[nodiscard]] virtual tl::expected<DatasetPtr, Error>
     GetDataByIds(const int64_t* ids, int64_t count) const {

src/algorithm/inner_index_interface.cpp

@@ -811,12 +811,22 @@ DatasetPtr
 InnerIndexInterface::GetDataByIdsWithFlag(const int64_t* ids,
                                           int64_t count,
                                           uint64_t selected_data_flag) const {
-    auto* inner_ids =
-        reinterpret_cast<InnerIdType*>(this->allocator_->Allocate(count * sizeof(InnerIdType)));
+    Vector<InnerIdType> inner_ids(allocator_);
+    return this->get_data_by_ids_with_flag(ids, count, selected_data_flag, inner_ids);
+}
+
+DatasetPtr
+InnerIndexInterface::get_data_by_ids_with_flag(const int64_t* ids,
+                                               int64_t count,
+                                               uint64_t selected_data_flag,
+                                               Vector<InnerIdType>& inner_ids) const {
+    CHECK_ARGUMENT(count >= 0, "count must not be negative");
+    inner_ids.clear();
+    inner_ids.reserve(static_cast<uint64_t>(count));
     {
         std::shared_lock lock(this->label_lookup_mutex_);
         for (int64_t i = 0; i < count; ++i) {
-            inner_ids[i] = this->label_table_->GetIdByLabel(ids[i]);
+            inner_ids.emplace_back(this->label_table_->GetIdByLabel(ids[i]));
         }
     }
     auto thread_count = static_cast<int64_t>(this->build_thread_count_);
@@ -832,8 +842,7 @@ InnerIndexInterface::GetDataByIdsWithFlag(const int64_t* ids,
         dataset->Float32Vectors(fp32_data);
         auto get_vec_func = [&](int64_t begin, int64_t end) {
             for (int64_t i = begin; i < end; ++i) {
-                auto inner_id = this->label_table_->GetIdByLabel(ids[i]);
-                this->GetVectorByInnerId(inner_id, fp32_data + i * this->dim_);
+                this->GetVectorByInnerId(inner_ids[i], fp32_data + i * this->dim_);
             }
         };
         if (this->thread_pool_ != nullptr) {
@@ -859,8 +868,7 @@ InnerIndexInterface::GetDataByIdsWithFlag(const int64_t* ids,
         dataset->AttributeSets(attribute_data);
         auto get_attr_func = [&](int64_t begin, int64_t end) {
             for (int64_t i = begin; i < end; ++i) {
-                auto inner_id = this->label_table_->GetIdByLabel(ids[i]);
-                this->GetAttributeSetByInnerId(inner_id, attribute_data + i);
+                this->GetAttributeSetByInnerId(inner_ids[i], attribute_data + i);
             }
         };
         if (this->thread_pool_ != nullptr) {
@@ -887,8 +895,8 @@ InnerIndexInterface::GetDataByIdsWithFlag(const int64_t* ids,
         dataset->ExtraInfos(extra_info)->ExtraInfoSize(static_cast<int64_t>(extra_info_size_));
         auto get_extra_info_func = [&](int64_t begin, int64_t end) {
             for (int64_t i = begin; i < end; ++i) {
-                auto inner_id = this->label_table_->GetIdByLabel(ids[i]);
-                this->extra_infos_->GetExtraInfoById(inner_id, extra_info + i * extra_info_size_);
+                this->extra_infos_->GetExtraInfoById(inner_ids[i],
+                                                     extra_info + i * extra_info_size_);
             }
         };
         if (this->thread_pool_ != nullptr) {
@@ -912,7 +920,6 @@ InnerIndexInterface::GetDataByIdsWithFlag(const int64_t* ids,
         memcpy(new_ids, ids, count * sizeof(int64_t));
         dataset->Ids(new_ids);
     }
-    this->allocator_->Deallocate(inner_ids);
     return dataset;
 }
 

src/algorithm/inner_index_interface.h

@@ -552,6 +552,12 @@ class InnerIndexInterface {
     }
 
 protected:
+    DatasetPtr
+    get_data_by_ids_with_flag(const int64_t* ids,
+                              int64_t count,
+                              uint64_t selected_data_flag,
+                              Vector<InnerIdType>& inner_ids) const;
+
     virtual MetadataPtr
     collect_streaming_header() const;
 

src/algorithm/pyramid/CMakeLists.txt

@@ -18,6 +18,8 @@ set (PYRAMID_SRCS
         pyramid.cpp
         pyramid_build_cache.cpp
         pyramid_index_node.cpp
+        pyramid_path_store.cpp
+        pyramid_paths.cpp
         pyramid_zparameters.cpp
 )