fix: caption images in the configured output language - #667
Open
b9bt5dp9hg-ship-it wants to merge 1 commit into
Open
fix: caption images in the configured output language#667b9bt5dp9hg-ship-it wants to merge 1 commit into
b9bt5dp9hg-ship-it wants to merge 1 commit into
Conversation
Image captions were always generated in English, even when the wiki's output language was set to something else — the caption prompt never mentioned a target language. On non-English wikis this produced English alt text inside otherwise localized source summaries, and the mismatch hurts retrieval (caption embeddings live in a different language than the queries). Fix: thread an optional outputLanguage through the caption pipeline. - vision-caption.ts: append a language instruction to both the no-context and context-aware prompts. Visible text in the image is still preserved verbatim in its original language. Absent or "auto" values leave the prompt untouched. - image-caption-pipeline.ts: pass options.outputLanguage through to captionImage. - ingest.ts: resolve the configured language at both caption call sites via getOutputLanguage() + getLanguagePromptName() — the same resolution buildLanguageDirective uses — so "auto" resolves from the source content instead of leaking the literal string "auto" into the prompt. Tests: language line present for both prompt variants, "auto"/absent fall back to the unmodified prompt, and the pipeline forwards the option. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
Image captions are always generated in English, regardless of the wiki's configured output language. The caption prompt (
CAPTION_PROMPT/buildCaptionPromptWithContext) never mentions a target language, so on non-English wikis the ingest pipeline produces English alt text inside otherwise localized source summaries. Beyond the visual inconsistency, this hurts retrieval: caption embeddings end up in a different language than the user's queries.Fix
Thread an optional
outputLanguagethrough the caption pipeline:vision-caption.ts— newcaptionLanguageInstruction()appends a language line ("Write the description in X. Preserve visible text verbatim in its original language.") to both the no-context and the context-aware prompt. Absent or"auto"values leave the prompts byte-identical to before, so existing caption caches and prompt regression tests are unaffected.image-caption-pipeline.ts—CaptionPipelineOptions.outputLanguageis passed through tocaptionImage.ingest.ts— both caption call sites resolve the configured language withgetOutputLanguage()+getLanguagePromptName(), the same resolutionbuildLanguageDirectivealready uses for page generation. This means"auto"resolves from the source content (instead of leaking the literal stringautointo the prompt), and captions follow the same language rules as the rest of the generated wiki.Visible text in images (labels, chart axes, OCR content) is still captured verbatim in its original language — only the descriptive prose follows the output language setting.
Tests
"auto"/ absentoutputLanguagefall back to the unmodifiedCAPTION_PROMPToutputLanguagetocaptionImagenpx tsc --noEmitclean, full mock suite green (1808 tests).🤖 Generated with Claude Code