fix: prevent silent polling death (zombie bot) — timeouts, bootstrap retries, liveness watchdog - #108
Closed
mondominator wants to merge 1 commit into
Closed
mondominator wants to merge 1 commit into
mondominator wants to merge 1 commit into
Conversation
…retries, liveness watchdog Three defects made the bot die silently while the process stayed alive (restart: always never fired): 1. getUpdates long-poll had no explicit timeouts — a silently dropped connection stalled polling forever (observed: Bad Gateway storm, then hours of zombie). 2. bootstrap_retries defaulted to 0 — a boot-time DNS failure killed the bot instead of retrying. 3. Nothing converted a dead polling loop into a process exit. Fixes: get_updates_* timeouts (10/30/10/10), run_polling(bootstrap_retries=-1), liveness watchdog thread that exits 75 after 3 failed checks so the container restart policy recovers, and sys.exit(1) on unhandled exceptions. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01XzUNvRCXML3DJe98KBCGkN
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
The bot can end up in a zombie state: the getUpdates polling loop dies, but the process stays alive — so
restart: alwaysnever fires and the bot silently stops responding until someone notices and manually restarts the container. I hit this twice in production:telegram.error.NetworkError: httpx.ConnectError: [Errno -3] Temporary failure in name resolution→ bot dead on arrival.run_polling()usesbootstrap_retries=0by default, so a single startup network error is fatal.Bad Gatewaystorm (5 errors logged through the error handler in ~20s), after which polling went silent for 2.5 hours with the process still running. With no explicitget_updates_*timeouts, a silently dropped long-poll connection can stall indefinitely.In both cases
docker psshows the container healthy andUp, while the bot is unresponsive in Telegram.Fix (three layers)
connect 10s / read 30s / write 10s / pool 10s) — a dropped connection now surfaces as an error PTB can retry, instead of hanging the polling task forever.run_polling(bootstrap_retries=-1)— startup network failures retry indefinitely instead of killing the bot (containers frequently race DNS/network at boot).application.running/updater.runningevery 60s (3-minute startup grace); after 3 consecutive failures it logs CRITICAL and exits 75 so the container restart policy can recover. Unhandled exceptions escapingrun_polling()nowsys.exit(1)for the same reason. A crashed bot that restarts in seconds beats a zombie that's dead for days.Notes
bootstrap_retriesdefault confirmed 0, allget_updates_*builder methods present.-d) behavior unchanged.🤖 Generated with Claude Code
https://claude.ai/code/session_01XzUNvRCXML3DJe98KBCGkN