feat: upstream sync 2026 07#21
Merged
qinxuye merged 1 commit intoJul 23, 2026
Merged
Conversation
|
Warning You have reached your daily quota limit. Please wait up to 24 hours and I will start processing your requests again! |
Port ~20 bug/security fixes and one new parser from upstream infiniflow/ragflow deepdoc/, re-implemented against this library's config-injection architecture (PdfModelConfig/TokenizerConfig, relative imports, ModelScope model distribution). Public API unchanged except one addition: deepdoc.EpubParser. Security (ragflow #12527): - harden zip extraction in MinerU and TCADP parsers: reject symlink, encrypted, absolute-path and traversal entries; stop zip-slip by resolving destinations against the extraction root; stream TCADP result downloads Vision: - rename StandardizeImag -> StandardizeImage so the dynamic getattr dispatch actually runs image standardization (#16785) - per-type layoutno namespaces for textless figure/equation placeholder boxes, preventing unrelated figure/equation crops from merging on scientific PDFs (#15873) - re-enable CID-pattern garbage filter for OCR text boxes (#13404) - English caption patterns (Figure/Table/Fig. N) in table caption detection (#15481) - env-configurable ONNX thread counts and opt-in GPU arena shrinkage (OCR_INTRA_OP_NUM_THREADS / OCR_INTER_OP_NUM_THREADS / OCR_GPUMEM_ARENA_SHRINKAGE) (#12777) Parsers: - excel: keep 0/False cells, skip empty rows, drop the spurious header-only chunk at exact chunk_rows multiples, binary-search the real row count for sheets with corrupt max_row (#16287/#15490/#13018) - html: parse bodyless fragments, fix h4 mapping to four hashes, CJK-aware oversized-block splitting that preserves original text, switch to html.parser (#16423/#13833/#16052/#14486) - markdown: adopt upstream protected-range extractor so fenced code and tables are never split by delimiters; GFM separators accept 1+ dashes; lone headers attach to the following section (#10896..#16319) - ppt: cached None-safe shape ordering (#13054) - utils: get_text treats empty bytes as binary input (#13196) PDF parser: - garbled-text detection with OCR fallback: PUA/unmapped-CID chars and subset-font encoding garbling at page and box level, skipped when the OCR alphabet cannot represent the text (#13404, #16958) - geometry-based word-space recovery, CJK-aware (#16958) - remove the hardcoded 299-page window (MAXIMUM_PAGE_NUMBER) and add chunked parse_into_bboxes with window-to-global remapping of boxes, position tags and positions (PDF_PARSER_PAGE_BATCH_SIZE) (#14382/#14385) - lazy img_np materialization in __ocr (#14385) - bounds-guard page indices in cropout and table/figure insertion (#12938/#12848) - VisionParser: absolute page numbering when from_page > 0 and fix the broken relative import of llm_adapter New parser: - EPUB parser: spine-order XHTML extraction delegating to HtmlParser, exported as deepdoc.EpubParser (#13650) Tests and docs: - tests/test_upstream_sync_regressions.py pins each ported fix without requiring model downloads - UPSTREAM_SYNC.md records the sync point, per-fix upstream references, and deliberately-skipped upstream changes (Go service pipeline, table auto-rotation, lazy_image refactors, 3-tuple outlines, cloud-API parsers) with reasons
kejhz653-stack
force-pushed
the
feat/upstream-sync-2026-07
branch
from
July 23, 2026 06:37
b5794e2 to
685085b
Compare
rogercloud
approved these changes
Jul 23, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.