Skip to content

fix(cli): harden UTF-8 terminal input handling#663

Merged
pancacake merged 2 commits into
HKUDS:devfrom
cmddirexit:fix/cli-utf8-terminal-input
Jul 23, 2026
Merged

fix(cli): harden UTF-8 terminal input handling#663
pancacake merged 2 commits into
HKUDS:devfrom
cmddirexit:fix/cli-utf8-terminal-input

Conversation

@cmddirexit

@cmddirexit cmddirexit commented Jul 22, 2026

Copy link
Copy Markdown

Summary

  • centralize UTF-8-aware reads in the shared CLI console input helper
  • temporarily enable termios.IUTF8 while reading UTF-8 input from a TTY
  • restore the original terminal attributes after input completes or is interrupted
  • keep the chat REPL alive when malformed terminal bytes still raise UnicodeDecodeError
  • protect both chat messages and ask_user prompts while leaving non-TTY, non-UTF-8, and unsupported platforms unchanged

Root cause

In canonical mode with IUTF8 disabled, erasing a multibyte character may remove only its final byte. For example, typing the three-byte UTF-8 character and pressing Backspace can leave the incomplete byte sequence e8 83. A strict UTF-8 stdin decoder then raises UnicodeDecodeError, terminating the chat REPL.

Reproduction

  1. Run stty -iutf8.
  2. Start deeptutor chat with strict UTF-8 stdin decoding.
  3. Type a Chinese character, press Backspace, and submit the line.

Tests

  • added a POSIX pseudo-terminal regression test that starts with IUTF8 disabled, enters , presses Backspace, and verifies both successful input and terminal-mode restoration
  • added a REPL test that injects UnicodeDecodeError and verifies the session remains usable
  • python -m pytest -q tests/cli (41 passed)
  • python -m ruff check deeptutor_cli/chat.py deeptutor_cli/common.py tests/cli/test_chat_cli.py tests/cli/test_chat_terminal.py
  • python -m ruff format --check deeptutor_cli/chat.py deeptutor_cli/common.py tests/cli/test_chat_cli.py tests/cli/test_chat_terminal.py

@pancacake
pancacake changed the base branch from main to dev July 23, 2026 14:23
@pancacake

Copy link
Copy Markdown
Collaborator

Thanks for your contribution!

@pancacake
pancacake merged commit 77bc735 into HKUDS:dev Jul 23, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants