feat: enrich RAG Markdown with stream provenance and cell metadata

Author: shps951023

docs/analytics-and-rag.md

@@ -126,7 +126,7 @@ The `miniexcel.rag-manifest/v1` JSON file records:
 
 Path exports implement `Iterator<Item = Result<RagChunk>>`; only one result chunk, repeated header context, and parser state need to be resident. `visit_rag_chunks_from_bytes` provides the equivalent callback form for browser and in-memory callers.
 
-`RagChunk::write_markdown` writes an independent GFM table directly to an I/O sink. `RagManifest::write_markdown_stream_end` can append an optional completion marker after the iterator is exhausted. Markdown is appendable and LLM-readable; JSONL remains canonical for exact typed metadata. See [Streaming Markdown and anydoc comparison](markdown-streaming.md) for the format, memory boundary, and benchmark harness.
+`RagManifest::write_markdown_stream_start` writes source, worksheet, range, and export provenance before iteration. `RagChunk::write_markdown` then writes an independent GFM table and bounded formula/style/number-format metadata directly to an I/O sink. `RagManifest::write_markdown_stream_end` can append an optional completion marker after the iterator is exhausted. Markdown is appendable and LLM-readable; JSONL remains canonical for exact typed metadata. See [Streaming Markdown and anydoc comparison](markdown-streaming.md) for the format, memory boundary, and benchmark harness.
 
 Hidden and very-hidden sheets are rejected by default. Call `with_allow_hidden_sheets(true)` only after an explicit privacy decision. Browser Lab exposes the same opt-in and performs all work locally in a Web Worker. It never sends workbook content to an external model.
 

docs/analytics-and-rag.zh-CN.md

@@ -126,7 +126,7 @@ println!("{}", export.manifest().source_sha256());
 
 路径导出实现 `Iterator<Item = Result<RagChunk>>`；内存中只需保留一个结果 chunk、重复的 header 上下文和 parser 状态。`visit_rag_chunks_from_bytes` 为浏览器和内存调用方提供等价的 callback 形式。
 
-`RagChunk::write_markdown` 将独立 GFM 表格直接写入 I/O sink。迭代器耗尽后，`RagManifest::write_markdown_stream_end` 可以附加可选完成标记。Markdown 可追加且便于 LLM 阅读；JSONL 仍是精确类型化 metadata 的规范表示。格式、内存边界和基准 harness 见[流式 Markdown 与 anydoc 对比](markdown-streaming.zh-CN.md)。
+`RagManifest::write_markdown_stream_start` 会在迭代前写入 source、worksheet、range 和 export provenance。随后，`RagChunk::write_markdown` 将独立 GFM 表格以及有界的 formula/style/number-format metadata 直接写入 I/O sink。迭代器耗尽后，`RagManifest::write_markdown_stream_end` 可以附加可选完成标记。Markdown 可追加且便于 LLM 阅读；JSONL 仍是精确类型化 metadata 的规范表示。格式、内存边界和基准 harness 见[流式 Markdown 与 anydoc 对比](markdown-streaming.zh-CN.md)。
 
 hidden 和 very-hidden sheet 默认会被拒绝。只有在作出明确隐私决定后，才调用 `with_allow_hidden_sheets(true)`。Browser Lab 提供相同的 opt-in，并在 Web Worker 中本地完成所有工作。它绝不会将 workbook 内容发送给外部模型。
 

docs/compatibility.md

@@ -33,7 +33,7 @@ The latest `calamine 0.36` and `rust_xlsxwriter 0.97` require Rust 1.88. The MVP
 | Typed `Query<T>` | `MiniExcel::query_as<T>()` | Streams rows and applies Serde mapping one row at a time |
 | Structure-preserving query | `MiniExcel::query_structured()` | Streams sparse rows with one-based coordinates, formulas, style IDs, and number formats |
 | Group/filter analytics | `MiniExcel::analyze_with_options()` | Versioned Rust extension; streams rows and retains only bounded group/evidence state |
-| RAG evidence export | `MiniExcel::export_rag()` | Versioned Rust extension; streams addressed JSONL-ready chunks and a source manifest |
+| RAG evidence export | `MiniExcel::export_rag()` | Versioned Rust extension; streams addressed JSONL-ready chunks, enriched GFM Markdown, and a source manifest |
 | `QueryRange` | `ReadOptions::with_start_cell()` / `with_end_cell()` | Inclusive A1 range for dynamic and typed reads |
 | `GetSheetNames` | `MiniExcel::get_sheet_names()` | Workbook order is preserved |
 | `GetSheetInformations` | `MiniExcel::get_sheet_info()` | Includes OOXML ID, order, name, type, visibility, and active state |
@@ -87,7 +87,7 @@ For typed writing, chrono values must use the matching MiniExcel helper (`serial
 
 Grouped analytics consume the dynamic row stream without retaining source rows. Memory additionally contains one aggregate state and bounded source-row evidence list per distinct group. `QueryPlan::max_groups` rejects the group that would exceed the configured limit. Result limits do not reduce group-state memory. Version 1 does not implement disk spill, sorted-input aggregation, or constant-memory high-cardinality grouping.
 
-Path RAG exports retain parser state, repeated header context, and one output chunk. Their manifest hashes the source file through a separate bounded read. Byte/WASM workflows avoid collecting source rows, but browser uploads inherently retain compressed XLSX bytes in WebAssembly memory; generated JSONL/Blob downloads also consume output-sized memory. Browser Lab runs these operations in a Web Worker for responsiveness, not as a claim of path-equivalent memory.
+Path RAG exports retain parser state, repeated header context, and one output chunk. Their manifest hashes the source file through a separate bounded read. Markdown includes stream-level source/sheet provenance and chunk-local formula/style/number-format metadata without retaining prior chunks. Byte/WASM workflows avoid collecting source rows, but browser uploads inherently retain compressed XLSX bytes in WebAssembly memory; generated JSONL, Markdown, and Blob downloads also consume output-sized memory. Browser Lab runs these operations in a Web Worker for responsiveness, not as a claim of path-equivalent memory.
 
 The backend makes two sequential, bounded-memory passes over the selected worksheet entry. The first records only the maximum used column and final explicitly declared row. This is required for MiniExcel-compatible stable dynamic schemas when legal files omit `<dimension>`, and to preserve style-only row elements like the .NET reader. The second pass emits rows. Worksheet XML and prior rows are never retained; memory consists primarily of shared strings, styles, parser buffers, the current row, and the bounded channel.
 
@@ -140,7 +140,7 @@ The contract covers only the current common surface: dynamic/typed path queries,
 | New-workbook `SaveAs` | Implemented and roundtrip-tested | Not yet |
 | Byte-array query/write for WASM | Implemented | Rust/browser tests |
 | Versioned grouped analytics | Rust research extension | No |
-| Addressed JSONL/manifest RAG export | Rust research extension | No |
+| Addressed JSONL/Markdown/manifest RAG export | Rust research extension | No |
 | Async APIs, DataReader, stream ownership | Deferred | No |
 | Insert/edit existing workbooks | Deferred | No |
 | CSV and legacy formats | Deferred | No |

docs/compatibility.zh-CN.md

@@ -33,7 +33,7 @@ Rust MVP 在统一的 `MiniExcel` facade 后实现最小但实用的 MiniExcel 
 | 类型化 `Query<T>` | `MiniExcel::query_as<T>()` | 流式处理 row，并逐行应用 Serde 映射 |
 | 保留结构的 query | `MiniExcel::query_structured()` | 流式输出稀疏 row，包含从 1 开始的坐标、公式、style ID 和 number format |
 | 分组/过滤分析 | `MiniExcel::analyze_with_options()` | 版本化 Rust 扩展；流式处理 row，只保留有界 group/evidence 状态 |
-| RAG 证据导出 | `MiniExcel::export_rag()` | 版本化 Rust 扩展；流式输出可转为带地址 JSONL 的 chunk 和源 manifest |
+| RAG 证据导出 | `MiniExcel::export_rag()` | 版本化 Rust 扩展；流式输出可转为带地址 JSONL 的 chunk、增强 GFM Markdown 和源 manifest |
 | `QueryRange` | `ReadOptions::with_start_cell()` / `with_end_cell()` | 动态和类型化读取使用包含端点的 A1 range |
 | `GetSheetNames` | `MiniExcel::get_sheet_names()` | 保持 workbook 顺序 |
 | `GetSheetInformations` | `MiniExcel::get_sheet_info()` | 包含 OOXML ID、顺序、名称、类型、visibility 和 active 状态 |
@@ -87,7 +87,7 @@ Rust MVP 在统一的 `MiniExcel` facade 后实现最小但实用的 MiniExcel 
 
 分组分析消费动态 row stream，而不保留源数据行。内存还会为每个不同 group 保存一个聚合状态和有界源 row evidence list。`QueryPlan::max_groups` 会拒绝超出配置 limit 的 group。结果 limit 不会减少 group-state 内存。版本 1 不实现磁盘 spill、已排序输入聚合或高基数分组的常量内存处理。
 
-路径 RAG 导出保留 parser 状态、重复 header 上下文和一个输出 chunk。manifest 通过单独的有界读取计算源文件 hash。Byte/WASM 工作流不会收集源数据行，但浏览器上传必然在 WebAssembly 内存中保留压缩后的 XLSX 字节；生成的 JSONL/Blob 下载也会消耗与输出大小相当的内存。Browser Lab 在 Web Worker 中运行这些操作是为了保持响应，而不是声称其内存与路径模式等价。
+路径 RAG 导出保留 parser 状态、重复 header 上下文和一个输出 chunk。manifest 通过单独的有界读取计算源文件 hash。Markdown 包含 stream 级 source/sheet provenance 以及 chunk 内 formula/style/number-format metadata，无需保留先前 chunk。Byte/WASM 工作流不会收集源数据行，但浏览器上传必然在 WebAssembly 内存中保留压缩后的 XLSX 字节；生成的 JSONL、Markdown 和 Blob 下载也会消耗与输出大小相当的内存。Browser Lab 在 Web Worker 中运行这些操作是为了保持响应，而不是声称其内存与路径模式等价。
 
 backend 对所选 worksheet entry 执行两次顺序、有界内存扫描。第一次只记录最大已使用 column 和最后一个明确声明的 row。这是为了在合法文件省略 `<dimension>` 时保持 MiniExcel 兼容的稳定动态 schema，并像 .NET reader 一样保留仅含 style 的 row element。第二次扫描输出 row。Worksheet XML 和先前 row 永远不会保留；内存主要由 shared string、style、parser buffer、当前 row 和有界 channel 构成。
 
@@ -140,7 +140,7 @@ Rust workflow 会在 Linux 和 Windows 上运行 Rust 契约。其 .NET parity j
 | 新 workbook `SaveAs` | 已实现并完成 roundtrip 测试 | 尚未 |
 | 用于 WASM 的字节数组 query/write | 已实现 | Rust/browser 测试 |
 | 版本化分组分析 | Rust 研究扩展 | 否 |
-| 带地址 JSONL/manifest RAG 导出 | Rust 研究扩展 | 否 |
+| 带地址 JSONL/Markdown/manifest RAG 导出 | Rust 研究扩展 | 否 |
 | Async API、DataReader、stream ownership | 延后 | 否 |
 | 插入/编辑现有 workbook | 延后 | 否 |
 | CSV 和旧格式 | 延后 | 否 |

docs/markdown-streaming.md

@@ -13,18 +13,29 @@ let options = ReadOptions::new().with_header_mode(HeaderMode::FirstRow);
 let mut export = MiniExcel::export_rag("book.xlsx", &options, &RagExportOptions::new())?;
 let stdout = std::io::stdout();
 let mut output = stdout.lock();
+export.manifest().write_markdown_stream_start(&mut output)?;
 for chunk in export.by_ref() {
     chunk?.write_markdown(&mut output)?;
 }
 export.manifest().write_markdown_stream_end(&mut output)?;
 # Ok::<(), miniexcel::Error>(())
 ```
 
-A chunk contains a start marker, a worksheet/range heading, an addressed GFM
-table, and an end marker. The first selected row is repeated as table context
-when header mode is enabled. Newlines and Markdown control characters from
-cells are escaped so source text cannot create rows, columns, headings, or raw
-HTML. Formula cells show both the cached value and formula text.
+A stream starts with source file name and SHA-256, worksheet name, one-based
+worksheet order, visibility, selected range, header mode, chunk limit, row
+limit, and formula-calculation policy. A chunk contains a start marker, a
+worksheet/range heading, an addressed GFM table, and an end marker. The first
+selected row is repeated as table context when header mode is enabled. Newlines
+and Markdown control characters from cells are escaped so source text cannot
+create rows, columns, headings, or raw HTML. Formula cells show both the cached
+value and formula text.
+
+Each chunk also includes a `Cell metadata` table when it contains formulas,
+nonzero style IDs, or non-`General` number formats. Rows identify the A1 cell,
+typed value category, formula, source style ID, and number format. This is
+source metadata rather than a visual reconstruction: GFM cannot reproduce
+Excel fonts, fills, borders, colors, or alignment, and MiniExcel does not expand
+the style table into those properties in Markdown.
 
 The final `miniexcel:stream-end` marker is optional. Its presence proves normal
 completion and records emitted rows, chunks, and deliberate truncation. Its
@@ -50,9 +61,15 @@ Parsing and chunk production still occur in a Web Worker.
 [firecrawl/anydoc](https://github.com/firecrawl/anydoc) has a broader goal: it
 converts many office formats into one consistent, compact Markdown document.
 For Excel, anydoc reads workbook bytes through calamine, constructs a complete
-document/table model, then renders one `String`. MiniExcel supports XLSX only
-but streams a selected worksheet and preserves row/A1/formula provenance for
-RAG ingestion.
+document/table model, then renders one `String`. It maps merged ranges into
+table spans, leaves covered GFM positions blank, and heuristically infers header
+rows from a bounded body sample. Its spreadsheet Markdown does not preserve
+formulas, Excel style IDs, number formats, workbook properties, comments,
+hyperlinks, or exact visual styling. MiniExcel supports XLSX only but streams a
+selected worksheet and preserves source/workbook/sheet, row/A1, formula, style
+ID, and number-format provenance for RAG ingestion. Header handling is explicit
+through `HeaderMode`; merged-cell spans are not yet represented in RAG
+Markdown.
 
 Run the Windows comparison harness from the repository root:
 

docs/markdown-streaming.zh-CN.md

@@ -11,14 +11,17 @@ let options = ReadOptions::new().with_header_mode(HeaderMode::FirstRow);
 let mut export = MiniExcel::export_rag("book.xlsx", &options, &RagExportOptions::new())?;
 let stdout = std::io::stdout();
 let mut output = stdout.lock();
+export.manifest().write_markdown_stream_start(&mut output)?;
 for chunk in export.by_ref() {
     chunk?.write_markdown(&mut output)?;
 }
 export.manifest().write_markdown_stream_end(&mut output)?;
 # Ok::<(), miniexcel::Error>(())
 ```
 
-每个 chunk 包含开始标记、worksheet/range 标题、带地址的 GFM 表格和结束标记。启用 header mode 时，首个选中 row 会在每个 chunk 中作为表格上下文重复出现。来自 cell 的换行和 Markdown 控制字符会被转义，因此源文本无法创建额外 row、column、标题或 raw HTML。公式 cell 会同时显示缓存值和公式文本。
+stream 开头会输出源文件名和 SHA-256、worksheet 名称、从 1 开始的 worksheet 顺序、可见性、选定范围、header mode、chunk 限制、row 限制和公式计算策略。每个 chunk 包含开始标记、worksheet/range 标题、带地址的 GFM 表格和结束标记。启用 header mode 时，首个选中 row 会在每个 chunk 中作为表格上下文重复出现。来自 cell 的换行和 Markdown 控制字符会被转义，因此源文本无法创建额外 row、column、标题或 raw HTML。公式 cell 会同时显示缓存值和公式文本。
+
+当 chunk 包含公式、非零 style ID 或非 `General` number format 时，还会附加 `Cell metadata` 表。每一行记录 A1 cell、value type、公式、源 style ID 和 number format。这些是源 metadata，而不是视觉重建：GFM 无法还原 Excel 的字体、填充、边框、颜色或对齐方式，MiniExcel 也不会在 Markdown 中把 style table 展开成这些属性。
 
 最终的 `miniexcel:stream-end` 标记是可选的。它存在时，可以证明 stream 正常完成，并记录已发出的 row、chunk 和有意截断状态；缺少该标记不会使此前完整的 chunk 失效。JSONL 仍是精确 value type、style 和 number format 的规范机器表示；Markdown 是便于 LLM 阅读的配套格式。
 
@@ -30,7 +33,7 @@ export.manifest().write_markdown_stream_end(&mut output)?;
 
 ## anydoc 对比
 
-[firecrawl/anydoc](https://github.com/firecrawl/anydoc) 的目标更广：将多种 office 格式转换成一致、紧凑的单个 Markdown 文档。对于 Excel，anydoc 通过 calamine 读取 workbook 字节，构建完整 document/table model，再渲染为一个 `String`。MiniExcel 只支持 XLSX，但它会流式处理选中的 worksheet，并为 RAG 摄取保留 row/A1/formula provenance。
+[firecrawl/anydoc](https://github.com/firecrawl/anydoc) 的目标更广：将多种 office 格式转换成一致、紧凑的单个 Markdown 文档。对于 Excel，anydoc 通过 calamine 读取 workbook 字节，构建完整 document/table model，再渲染为一个 `String`。它会把 merged range 映射为 table span，在 GFM 中将被覆盖的位置留空，并根据有限的 body sample 启发式推断 header row。其 spreadsheet Markdown 不保留公式、Excel style ID、number format、workbook properties、comment、hyperlink 或精确视觉样式。MiniExcel 只支持 XLSX，但它会流式处理选中的 worksheet，并为 RAG 摄取保留 source/workbook/sheet、row/A1、formula、style ID 和 number format provenance。Header 通过 `HeaderMode` 明确指定；RAG Markdown 尚未表示 merged-cell span。
 
 从仓库根目录运行 Windows 对比 harness：
 

miniexcel-cli/src/main.rs

@@ -289,6 +289,9 @@ fn rag_export(arguments: RagExportArgs) -> CliResult<()> {
     let mut markdown_output = markdown_file.as_mut().map(|file| BufWriter::new(file.as_file_mut()));
 
     let mut export = MiniExcel::export_rag(&arguments.file, &options, &export_options)?;
+    if let Some(output) = &mut markdown_output {
+        export.manifest().write_markdown_stream_start(&mut *output)?;
+    }
     for chunk in export.by_ref() {
         let chunk = chunk?;
         if let Some(output) = &mut jsonl_output {

miniexcel-wasm/src/lib.rs

@@ -235,6 +235,12 @@ fn export_rag(
             chunks.push(chunk.clone());
             Ok(())
         })?;
+    let mut markdown_start = Vec::new();
+    manifest.write_markdown_stream_start(&mut markdown_start)?;
+    let markdown_start_len = markdown_start.len();
+    chunks_markdown.reserve(markdown_start_len);
+    chunks_markdown.extend_from_slice(&markdown_start);
+    chunks_markdown.rotate_right(markdown_start_len);
     manifest.write_markdown_stream_end(&mut chunks_markdown)?;
     let chunks_markdown =
         String::from_utf8(chunks_markdown).expect("Markdown serializer emits UTF-8");