Skip to content

Enhance East Asian width handling and text input buffer functionality#335

Merged
shibayan merged 1 commit intomasterfrom
support-zwj
Mar 9, 2026
Merged

Enhance East Asian width handling and text input buffer functionality#335
shibayan merged 1 commit intomasterfrom
support-zwj

Conversation

@shibayan
Copy link
Owner

@shibayan shibayan commented Mar 9, 2026

This pull request significantly improves how the code handles complex Unicode text elements, such as emojis and combined characters, in both width calculation and text input buffer operations. The changes ensure accurate cursor movement, deletion, and word navigation for grapheme clusters, including multi-codepoint emojis and regional indicators. Comprehensive tests were added to validate these behaviors.

Unicode and emoji handling improvements:

  • Enhanced EastAsianWidth.GetWidth to correctly calculate string width for emojis, regional indicators, zero-width joiners, and other complex Unicode sequences, including handling zero-width and emoji-modifier codepoints. [1] [2]
  • Added new test cases in EastAsianWidthTests.cs for zero-width joiner, emoji sequences, and combined emojis to ensure correct width calculation.

Text input buffer enhancements:

  • Refactored TextInputBuffer to operate on Unicode text elements (grapheme clusters) rather than individual code units, improving correctness for insertion, deletion, backspacing, and cursor movement.
  • Implemented efficient text element boundary tracking and navigation, including methods for moving cursor and deleting by grapheme clusters, and for word-based operations.

Testing for complex text input scenarios:

  • Added comprehensive tests in TextInputBufferTests.cs for backspace, delete, move backward/forward, and word navigation with complex emojis and combined characters, ensuring robust handling of these cases. [1] [2] [3] [4] [5]

@shibayan shibayan requested a review from Copilot March 9, 2026 11:08
@shibayan shibayan self-assigned this Mar 9, 2026
@shibayan shibayan added the enhancement New feature or request label Mar 9, 2026
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR improves Unicode correctness in Sharprompt’s terminal rendering and interactive editing by switching width calculation and text input buffer operations to be grapheme-cluster aware (handling emoji ZWJ sequences, keycaps, modifiers, and regional indicator flags).

Changes:

  • Updated EastAsianWidth.GetWidth to iterate by text elements (grapheme clusters) and to treat certain Unicode categories as zero-width, with added emoji/sequence-aware width rules.
  • Refactored TextInputBuffer cursor movement and deletion/word operations to act on text elements instead of UTF-16 code units.
  • Added new tests covering complex emoji/grapheme scenarios for both width calculation and text buffer operations.

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 3 comments.

File Description
Sharprompt/Internal/TextInputBuffer.cs Refactors editing/navigation logic to use grapheme-cluster boundaries tracked via StringInfo.ParseCombiningCharacters.
Sharprompt/Internal/EastAsianWidth.cs Changes width calculation to be text-element based and adds special handling for emoji sequences/zero-width code points.
Sharprompt.Tests/TextInputBufferTests.cs Adds test coverage for complex emoji grapheme clusters across editing and navigation operations.
Sharprompt.Tests/EastAsianWidthTests.cs Adds tests for ZWJ, emoji sequences, keycaps, modifiers, and regional indicator flags.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@shibayan shibayan merged commit e2c9292 into master Mar 9, 2026
9 checks passed
@shibayan shibayan deleted the support-zwj branch March 9, 2026 11:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants