Conversation
Adds a "Speak text" PostGrabAction that reads OCR output aloud using Windows.Media.SpeechSynthesis. Speaks the final transformed text after all other actions run in FullscreenGrab; in GrabFrame, speaks only when the captured text changes to avoid repeating on every OCR tick. - ITtsEngine interface for future engine swappability - WindowsSpeechEngine wraps WinRT SpeechSynthesizer + MediaPlayer - TtsService queues utterances via Channel<string> so new text waits rather than interrupting in-progress speech - TtsSpeakWordLimit setting (default 100) truncates long captures; configurable in General Settings - PostGrabActionManager: new SpeakText_Click action at order 6.6 - GrabFrame: speaks on text change when action is checked - Tests: count updated to 6, "Speak text" assertion, fire-and-forget test Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Adds SpeakInsteadOfToast setting: when enabled, Text Grab speaks the captured text aloud rather than chiming a notification toast. Defaults to enabled. Includes: - Stop-speaking button in GrabFrame to cancel playback mid-sentence - TTS drain-on-shutdown fix so the queue empties cleanly on exit - Stop() method on TtsService to flush the queue and cancel speech
Adds a dedicated Voice Output page in Settings containing: - Voice picker populated from SpeechSynthesizer.AllVoices - Word limit setting - Speak-instead-of-notification toggle (moved from General Settings) - Preview button to hear the selected voice WindowsSpeechEngine now applies the saved TtsVoiceName before synthesising. TtsVoiceName setting added (empty = system default).
Introduce a new user-scoped boolean setting, WindowsAiDescriptionEnabled, to control Windows AI description features. The setting defaults to False and is added to App.config, Settings.settings, and Settings.Designer.cs.
Added WindowsAiDescriptionLang implementing ILanguage for the new "Windows AI Description" language kind. Introduced GeneratedOcrLinesWords, GeneratedOcrLine, and GeneratedOcrWord classes to generate OCR line/word models from text and bounding boxes. Updated HistoryInfo to support the new language kind.
Introduce WindowsAiDescriptionLang for image captioning via Windows AI. Update LanguageKind, LanguageService, and CaptureLanguageUtilities to support selection and caching. Route OCR/image description requests through new async methods in OcrUtilities and WindowsAiUtilities. Add device capability checks, content moderation, and async polling for image description. Exclude WindowsAiDescriptionLang from table output. Improve resource cleanup and error handling.
Improve test coverage for WindowsAiDescriptionLang integration: - Save/restore WindowsAiDescriptionEnabled for isolation - Test inclusion/exclusion in CaptureLanguageUtilities - Assert correct tag/kind handling in LanguageService - Verify fallback and HistoryInfo behavior with new language
Introduce WindowsAiDescriptionLang support in LanguagePicker and filter it from the language list. Add a toggle in LanguageSettings to enable/disable Windows AI Description as a language option, with explanatory text. Refactor settings initialization to use a unified loadingLanguageSettings flag and update event handlers accordingly. Implement methods to load and persist the new setting, and ensure language caches are invalidated when toggled.
Add the five Microsoft.Recognizers.Text.* packages (with BUILT-WITH
attribution) and a fixed BuiltInRecognizer catalog -- numbers, ordinals,
percentages, ages, currencies, dimensions, temperatures, dates/times, phone
numbers, emails, URLs, IPs, GUIDs, and booleans -- plus RecognizerExecutor for
matching and applying them (resolved value or matched text, first/last/all/nth).
GrabTemplate gains {r:Name:mode[:value|text][:separator]} placeholders
(TemplateRecognizerMatch), resolved by GrabTemplateExecutor alongside the
existing {p:} regex patterns.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Introduce PatternItem -- a single user-facing "pattern" that wraps either a saved
StoredRegex or a built-in recognizer -- and PatternExecutor, which dispatches
HasMatch/GetMatches/Apply to the regex or recognizer engine. This lets every UI
surface treat the two kinds uniformly. PatternItem also builds the grouped
("Saved Patterns" / "Smart Patterns") choice lists used by the combo pickers.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Expose saved regexes and built-in recognizers together everywhere, split into
"Saved Patterns" and "Smart Patterns" subsections:
- Template inline {...} picker: one Patterns group (InlinePickerItem.Kind drives
whether selection emits {p:} or {r:}); {p:}/{r:} syntax unchanged.
- Edit Text Window: "Apply Pattern" / "Apply Pattern Per Line" list both kinds and
transform in place via PatternExecutor.
- Grab Frame: "Search by Pattern" submenu (saved regexes searchable for the first
time); Quick Simple Lookup and Find & Replace use a unified Patterns combo
(saved regex loads into the find box so replace still works; recognizers are
find-only).
Combo pickers use PatternChoice data items with DisplayMemberPath and
non-selectable subsection headers, fixing the selection-box width/height clipping
that came from using ComboBoxItem objects as data. User-visible "Recognizer"
wording is now "Pattern".
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
A reusable search input used across the search surfaces: a text box with a placeholder and clear button, a removable smart-pattern chip, a WPF-UI-style accent focus underline on the whole control, a regex toggle rendered as a split-button whose chevron opens a scrollable pattern flyout (saved regexes + smart recognizers), and an optional exact-match toggle. Adds the ToggleSplitSegment and SplitSegmentButton styles the split-button uses. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
… Frame Replaces each window's hand-built search box, regex control, and pattern picker with the shared SearchBar and routes searching through its SearchChanged event: - Quick Simple Lookup: swaps its text box, regex toggle, and pattern combo for the control; regex validity now flows through SetRegexValidity. - Find & Replace: the find box becomes a SearchBar; the "Match Exactly" switch moves into the box as a toggle (wired via ExactMatchChanged), and a picked recognizer can be narrowed by typed text. Adds SetFindText for external callers. - Grab Frame: the regex, exact-match, and "search by pattern" options that were hidden in the text box context menu become visible controls, and the status bar grows to fit the taller control. Retargets RegexManager and EditTextWindow to SetFindText and drops the now-unused PatternItem combo helpers. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Registers a Ctrl+Shift+F key gesture in the Edit Text Window that opens Find & Replace, and shows the shortcut on the menu item. Ctrl+F stays mapped to Fullscreen Grab. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Expanded GetDefaultPatterns summary to clarify exclusion of patterns already handled by built-in recognizers (emails, phones, URLs, IPs, GUIDs, dates, times, currency, numbers). Removed these patterns from the defaults, retaining only credit card, hex color, SSN, and US zip code regexes.
Recognizers: unify saved regexes and smart recognizers as one "Patterns" concept
Introduce a new user-scoped boolean setting, WindowsAiDescriptionEnabled, to control Windows AI description features. The setting defaults to False and is added to App.config, Settings.settings, and Settings.Designer.cs.
Added WindowsAiDescriptionLang implementing ILanguage for the new "Windows AI Description" language kind. Introduced GeneratedOcrLinesWords, GeneratedOcrLine, and GeneratedOcrWord classes to generate OCR line/word models from text and bounding boxes. Updated HistoryInfo to support the new language kind.
Introduce WindowsAiDescriptionLang for image captioning via Windows AI. Update LanguageKind, LanguageService, and CaptureLanguageUtilities to support selection and caching. Route OCR/image description requests through new async methods in OcrUtilities and WindowsAiUtilities. Add device capability checks, content moderation, and async polling for image description. Exclude WindowsAiDescriptionLang from table output. Improve resource cleanup and error handling.
Improve test coverage for WindowsAiDescriptionLang integration: - Save/restore WindowsAiDescriptionEnabled for isolation - Test inclusion/exclusion in CaptureLanguageUtilities - Assert correct tag/kind handling in LanguageService - Verify fallback and HistoryInfo behavior with new language
Introduce WindowsAiDescriptionLang support in LanguagePicker and filter it from the language list. Add a toggle in LanguageSettings to enable/disable Windows AI Description as a language option, with explanatory text. Refactor settings initialization to use a unified loadingLanguageSettings flag and update event handlers accordingly. Implement methods to load and persist the new setting, and ensure language caches are invalidated when toggled.
…-Grab into WinAI-description
Add overloads to GetTextDescriptionWithWinAI for System.Drawing.Bitmap input and CancellationToken support. Use AsTask to enable cancellation of DescribeAsync, remove manual polling, and improve exception handling to propagate OperationCanceledException and log other errors. Temp files are now cleaned up after use.
Introduce a new user flow for Windows AI image description in fullscreen grab mode. The overlay now freezes the selected region as a background, shows a spinner, and after 2 seconds offers Cancel/Re-grab options. If the AI description fails or is empty, a "Send to Grab Frame" option is provided. Refactor PreviousGrabWindow and FullscreenGrab to support this asynchronous, user-driven workflow and improve code organization.
Replace the flat black strip with a centered rounded panel using the app background, teal border, and drop shadow like the fullscreen grab toolbar. Buttons now use the TealColor style with icon + label content, and the bar slides up from the bottom edge with a fade on first reveal. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- Use C# collection expressions for English word array in WindowsAiUtilities - Return empty string (not error) if WinAI unavailable for image description - Ignore single clicks for AI Description in fullscreen grab, reset selection instead - Cancel grab if overlay closes without user choice to avoid orphaned tasks - Default TextContent to empty string if OCR result is null
Adds ShowSuccessAsync to PreviousGrabWindow for awaiting overlay close, ensuring text insertion occurs after focus is released. Refactors insertion logic in FullscreenGrab to prevent focus issues, threads cancellation tokens in WindowsAiUtilities, and cleans up redundant code.
Adds a ToggleButton to PreviousGrabWindow for sending OCR results to an Edit Text Window, with visibility and checked state managed during OCR. Introduces a SendToEditTextToggled event and syncs the toggle state across all fullscreen grab windows. Updates running/failed state logic to show or hide the toggle as appropriate.
WinAI Descriptions
Introduce a new .tggf file format for saving and reopening Grab Frame sessions, bundling image, word borders, and settings. Update the MSIX manifest and add registry logic for file association in both packaged and unpackaged installs. Implement GrabFrameFileUtilities for ZIP-based read/write, update App and GrabFrame to handle open/save, and extend file dialogs and filters. Add unit tests for round-trip, association, and error handling. Closes #229
Users can now select the Grab Frame border style (theme, high contrast, or custom color) via the settings UI and context menu. Choices are saved in user settings and applied live, with UI updates reflecting the current selection. This improves accessibility and customization of the Grab Frame appearance. Closes #579 Closes #587
Ports the furigana-stripping feature from the previous better-cjk work onto dev's OCR pipeline. Small ruby-text characters rendered above the main text are detected by height and vertical position and removed before the CJK words are merged in GetTextFromOcrLine. Adds a deterministic unit test using OCR test doubles rather than the prior branch's flat-word-border approach, which depended on the now-removed PostOcrUtilities implementation. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Covers the FilterFurigana heuristic directly (empty/single/uniform input, overlap and vertical-position requirements, the "larger word below" requirement, and the word-length cutoff) plus end-to-end language gating through BuildTextFromOcrLines for Japanese, Chinese, and space-joining languages. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Added a user setting to enable/disable furigana (ruby text) removal from Japanese and Chinese OCR results. Updated LanguageSettings UI with a toggle, explanation, and info note. Synced toggle state with settings and cache invalidation logic.
Comprehensive Japanese OCR test coverage with a new furigana-annotated image. Added geometry-based line ordering and furigana line filtering to OcrUtilities, with opt-in removal via settings. Updated test project to include the new image and validated all new behaviors with unit tests.
Better Japanese post-processing
Added two new user-scoped Boolean settings: HdrCaptureCorrection and HdrBorderlessGranted, both defaulting to False. Updated App.config, Settings.settings, and Settings.Designer.cs to support these options as strongly-typed properties.
Introduce DisplayHdrInfo, HdrScreenCapture, and HdrToneMapper under Text_Grab.Utilities.Hdr. These components enable accurate detection of per-monitor HDR state, high-precision screen capture from HDR displays, and correct tone-mapping to SDR bitmaps. Addresses washed-out capture issues on HDR monitors by leveraging DXGI, Direct3D 11, and Windows.Graphics.Capture APIs.
Integrate HDR screen region capture with tone mapping fallback to GDI when needed. Add Vortice.Direct3D11/DXGI dependencies for HDR support. Refactor ImageMethods to centralize capture logic. Add HdrToneMapper unit tests to validate tone mapping behavior.
Added an "HDR Displays" section to GeneralSettings with a toggle for HDR capture correction and a button to check/grant borderless capture permissions. Updated code-behind to handle toggle state, update settings, and display permission status to the user. Added explanatory text for new options.
Added Bash(git log *) to settings.local.json to permit execution of git log commands via Bash, expanding the set of allowed command patterns.
Added Bash(gh pr view *) and Bash(gh pr diff *) to the allowlist in settings.local.json, enabling use of GitHub CLI commands for viewing and diffing pull requests.
Refactored HDR screen capture to use a process-wide shared D3D11 device, context, and WinRT device for efficiency and reliability. Reduced frame timeout to 250ms to avoid UI stalls. Added helpers for device lifecycle management and improved error handling to recover from device loss. Ensured proper resource cleanup and event unsubscription.
Support HDR Capture
Introduced SettingsCard and SettingsExpander custom controls for consistent settings UI, modeled after Windows Community Toolkit patterns. Refactored ShortcutControl.xaml to use a grid layout for improved alignment and visual consistency; ErrorText is now wrapped, collapsed by default, and spans both columns.
Added SettingsStyles.xaml with styles for settings section headers and page titles. Integrated the new resource dictionary into App.xaml for consistent settings UI appearance.
Refactored all settings pages to use new SettingsCard and SettingsExpander controls, replacing legacy layouts with a more modern, consistent, and accessible design. Grouped related controls with icons, headers, and descriptions for clarity. Updated section headers and page titles for uniformity. No changes to core logic—UI/UX improvements only.
Capture region bitmap before showing overlays (fixes #662). Pass pre-captured bitmap to OCR routines to avoid UI artifacts in OCR input and history. Update method signatures and call sites to support optional pre-captured bitmap.
Added "Bash(git fetch *)" to the allow list in settings.local.json, enabling the use of git fetch in Bash scripts. No other changes were made.
Add tracking of active spreadsheet cell editor and selected text in EditTextWindow. Implement GetSpreadsheetSelectedText to retrieve selection from the cell editor, last known selection, or cell value. Update Find/Replace to use this logic in spreadsheet mode. Handle PreparingCellForEdit to attach selection tracking and ensure cleanup on edit end and selection changes. closes #663
Introduced ColumnSplitUtilities with support for delimiter, regex, and fixed-length cell splitting, including splitter handling options and robust error handling. Added thorough unit tests covering all split modes and edge cases.
Introduces SplitColumnWindow with UI and logic to split spreadsheet cell contents by delimiter, regex pattern, or fixed length. Features include pattern picker, regex extraction from selection, splitter handling options, live preview, and integration with EditTextWindow for applying splits to selected cells.
Added "Split into Columns…" to the spreadsheet context menu. Implemented handlers to open a split dialog and apply user-defined splitting of selected cell contents into multiple columns, resizing the table as needed.
Updated NCalcAsync, PdfPig, and Microsoft.NET.Test.Sdk to latest versions. Added NuGet.CommandLine (v7.6.0) as a development dependency in all projects. No code changes, only package reference updates.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Microsoft.Recognizers.Text erroneously declares a dependency on the NuGet.CommandLine build tool, which pulls nuget.exe into the graph. Keep a single ExcludeAssets="all" reference in the app project so it satisfies resolution without flowing any assets into the build output, and drop the stray references from the test and packaging projects. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The save blanked WordBorderInfoJson, WordBorderInfoFileName, and ImagePath directly on the supplied instance to keep the serialized metadata clean. That silently corrupts any caller that passes a shared/live HistoryInfo rather than a throwaway from AsHistoryItem(). Serialize a shallow copy instead so the caller's object is left untouched, and add a test asserting the passed-in fields (including the ImageContent reference) are unchanged. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
… type The DateTime "values" branch cast the resolution entry to the library's concrete IEnumerable<Dictionary<string,string>>. A future Recognizers-Text version storing it as Dictionary<string,object> would fail the cast and silently fall back to the matched text. Read the collection structurally (any non-string IEnumerable of dictionaries, string- or object-valued) instead. Add FormatResolvedValue tests over synthetic ModelResults pinning both the current and object-valued shapes, plus real datetime recognizer tests that lock the live library shape, so a package bump surfaces loudly. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The hand-maintained versions had already drifted from the csproj (WPF-UI, Magick.NET, PdfPig, and others). Remove the column and point readers to Text-Grab.csproj / Tests.csproj as the single source of truth, and add the Vortice.Direct3D11 / Vortice.DXGI packages introduced for HDR capture. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
PR Classification
New feature and major UI/UX enhancement: unified pattern/recognizer search, voice output, HDR capture correction, and improved settings.
PR Summary
This PR introduces unified "Smart Pattern" (recognizer) and regex search across the app, adds voice output (TTS) features, supports HDR-aware screen capture, and modernizes the settings UI. It also improves Grab Frame and Quick Simple Lookup workflows and adds .tggf file support.