Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
25 commits
Select commit Hold shift + click to select a range
59e7340
feat(agentos): add first-class PocketJS app runtime
siwei-yuan Aug 10, 2026
5f863f1
chore(pocketjs): pin AgentOS to upstream main
siwei-yuan Aug 10, 2026
8f294da
feat(agentos): adopt PocketJS net for Exa
siwei-yuan Aug 10, 2026
bbd1d64
fix(agentos): provision Robinhood and bound Exa storage
siwei-yuan Aug 10, 2026
ca3335c
feat(ui): add keyboard press feedback
siwei-yuan Aug 10, 2026
2fd0182
fix(esp32): reduce touch polling latency
siwei-yuan Aug 10, 2026
c5861b9
feat(agent): add DeepSeek V4 backend and fix chart rendering
siwei-yuan Aug 10, 2026
c6a1191
feat(ui): expand Pi Design and improve ESP readability
siwei-yuan Aug 10, 2026
2477228
feat(agentos): add deferred app tools and persistent workers
siwei-yuan Aug 11, 2026
d9f26d7
refactor(agentos): remove legacy runtime paths
siwei-yuan Aug 11, 2026
aed2a0e
refactor: remove desktop full-pi runtime
siwei-yuan Aug 11, 2026
3445409
fix(agent): accept long bootstrap prompts safely
siwei-yuan Aug 11, 2026
861cff4
refactor(robinhood): keep tool descriptions policy-free
siwei-yuan Aug 11, 2026
03c35b1
fix(esp32): retry stale model connection once
siwei-yuan Aug 11, 2026
c031e18
fix(ui): wrap and truncate next wake preview
siwei-yuan Aug 11, 2026
9b37c75
fix(robinhood): align catalog descriptions with exposed tools
siwei-yuan Aug 11, 2026
83192f7
fix(esp32): drop late hosted RPC responses
siwei-yuan Aug 11, 2026
6b69424
fix(robinhood): reconnect stalled read-only requests
siwei-yuan Aug 11, 2026
9e29ec5
fix(agentos): keep app database work responsive
siwei-yuan Aug 11, 2026
cb46e40
refactor(agentos): select apps at firmware build time
siwei-yuan Aug 11, 2026
af7c94a
fix(agentos): keep streaming runtime responsive
siwei-yuan Aug 11, 2026
8ab03a7
Improve long-text rendering and Exa scrolling
siwei-yuan Aug 12, 2026
bceb641
refactor(agentos): enforce app firmware boundaries
siwei-yuan Aug 12, 2026
64a1fe3
Merge remote-tracking branch 'upstream/main' into codex/unified-hosts
siwei-yuan Aug 12, 2026
64f951e
docs: define Pocket Pi as a device runtime
siwei-yuan Aug 12, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 6 additions & 4 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,9 @@ jobs:
rustup component add clippy
rustup default stable

- name: Bun
uses: oven-sh/setup-bun@v2

- name: Cache cargo
uses: actions/cache@v4
with:
Expand All @@ -39,9 +42,8 @@ jobs:
- name: Build
run: cargo build --workspace --all-targets

# Runs the unit + module-system suite. The Path B integration tests are
# #[ignore]: they need the ~13 MB esbuild bundle (git-ignored; built with
# `node js/build-pi-full.mjs`) and, for the turn tests, an API key + proxy —
# so they're driven locally, not in CI.
- name: Test
run: cargo test --workspace

- name: App text behavior tests
run: bun test apps/_shared/text.test.ts
6 changes: 0 additions & 6 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -10,9 +10,3 @@ node_modules
__pycache__/
*.py[cod]
artifacts/screenshots/

# The raw full-pi bundle (~9 MB) is a build intermediate — rebuild with
# `node js/build.mjs`. The gzip (pi-full.bundle.js.gz, ~1.8 MB) IS committed: the
# crate embeds it (include_bytes!) so a plain `cargo build` — including from the
# vendored submodule in cat — works with only Rust, no Node.
crates/pocket-pi/js/pi-full.bundle.js
182 changes: 62 additions & 120 deletions ARCHITECTURE.md
Original file line number Diff line number Diff line change
@@ -1,143 +1,85 @@
# Pocket Pi architecture

Pocket Pi is one runtime family with two agent profiles and three hosts.
Pocket Pi is a complete Agent-native runtime for embedded and dedicated
devices. The Agent is a resident system actor with a persistent workspace,
native capabilities, schedules and Agent-native Apps—not a desktop application
or a generic Agent SDK.

| Run mode | Host | Pocket Pi profile | Purpose |
|---|---|---|---|
| Native Mac | `hosts/macos` | full `pi-coding-agent` | Normal desktop Pocket Pi |
| ESP32 simulator on Mac | `hosts/esp32-p4-sim` | embedded `pi-agent-core` + device UI/tools | Fast development of embedded contracts and product flows |
| Physical ESP32-P4 | `firmware/esp32-p4` | embedded `pi-agent-core` + device UI/tools | Real PocketJS/QuickJS Agent on the board |
The current implementation has one supported hardware target and one companion
development tool:

These are not three products. They are three compositions of Pocket Pi. The
simulator and physical firmware share the embedded runtime, UI, tool contracts
and interaction semantics. The simulator may use simpler host implementations;
physical hardware remains the final acceptance target.
| Role | Composition | Status |
| --- | --- | --- |
| Reference hardware | `firmware/esp32-p4` | ESP32-P4 is the first fully supported Pocket Pi target |
| Development simulator | `hosts/esp32-p4-sim` | Runs the ESP32-P4 product contracts on macOS; not a desktop product or hardware target |

```text
full desktop profile embedded product profile
crates/pocket-pi crates/pocket-pi-embedded
│ │
hosts/macos crates/pocket-pi-device-ui
┌────────────┴────────────┐
│ │
firmware/esp32-p4 hosts/esp32-p4-sim
```
Both compositions use the same resident `pi-agent-core` System App and PocketJS
App bundles. The simulator substitutes development adapters; only the physical
composition proves hardware behavior.

The authoritative detailed design is
[`docs/agentos-architecture.md`](docs/agentos-architecture.md).

## Ownership

- `crates/pocket-pi` runs the full, unmodified `pi-coding-agent` with its
desktop Node/Web compatibility layer.
- `crates/pocket-pi-embedded` runs the bounded upstream `pi-agent-core` loop.
Native host traits provide model and tool capabilities.
- `crates/pocket-pi-device-ui` is the single source for the 720x1280 PocketJS
draw list, fonts, touch hit map, Chat, Workspace browser, keyboard, message
reader, device Settings and system status. The host supplies the mounted
workspace root.
- `crates/pocket-pi-agentos` owns App catalog, runtime lifecycle, foreground
selection, schedules, App Tool routing, and App-scoped FS/SQLite mounts.
- `crates/pocket-pi-app-pack` composes the build-selected App artifacts without
moving their product logic into the AgentOS runtime or host adapters.
- `crates/pocket-pi-embedded` provides the bounded JavaScript Agent Loop bridge.
In AgentOS hosts, the loop is loaded from the Pi Agent System App release into
the same PocketJS Guest as its Root View.
- `apps/pi-agent` owns the Root View and Agent Loop release artifacts.
- `apps/robinhood` and `apps/exa` own their Tools, Tasks, SQLite schemas, and
PocketJS Views.
- `crates/pocket-pi-tools` owns portable native workspace, bounded shell, time,
device, and Agent schedule Tools.
- `crates/pocket-pi-protocols` owns model/provider transport protocols.
- `crates/pocket-pi-tools` owns the portable native ESP tool registry:
filesystem tools, bounded bash, workspace context, time and schedules.
- Each host is a composition root. It connects the embedded Agent, shared UI,
filesystem, input, display and model adapter.

Dependencies point inward: hosts depend on the runtime, UI and protocols. The
runtime and UI do not depend on a host. External products can populate a UI
projection or register a native tool without putting provider clients in core.

## Repository map

```text
crates/pocket-pi/ full desktop pi-coding-agent runtime
crates/pocket-pi-embedded/ bounded pi-agent-core guest + native host traits
crates/pocket-pi-tools/ portable native workspace, shell, time and schedule tools
crates/pocket-pi-protocols/ provider request/response and streaming codecs
crates/pocket-pi-device-ui/ shared PocketJS embedded UI and interaction state
hosts/macos/ desktop composition root
hosts/esp32-p4-sim/ macOS implementation of the embedded host adapters
firmware/esp32-p4/ ESP-IDF hardware composition root and adapters
tools/uart_bridge/ Mac Codex/Claude streaming adapters
tools/uart-model-bridge.py thin UART framing and provisioning CLI
```

The split follows ownership, not product features. A native tool is implemented
once in `pocket-pi-tools`; a provider codec belongs in `pocket-pi-protocols`;
hardware access stays in a host. Optional applications such as Exa or
Robinhood should be separate tool/plugin adapters and must not become
dependencies of the embedded runtime or shared UI.

## ESP32 and simulator parity

The physical ESP32-P4 firmware and macOS simulator compile the same:
- Hosts own hardware, transport, credentials, and rendering adapters.

- `pocket-pi-embedded` Agent runtime;
- `pocket-pi-device-ui` Rust source and exact font atlases;
- `ScreenState::handle_tap` coordinate hit map;
- 720x1280 PocketJS draw-list viewport.
There is no legacy Rust product UI or general-purpose desktop runtime. The
simulator and ESP32-P4 render the same
PocketJS App bundles at a 720x1280 logical viewport. Rust firmware supplies the
display/touch driver and renders the selected App's DrawList.

The simulator maps a mouse pointer into the same 720x1280 coordinates used by
the touch controller and calls the same `handle_tap` method. It substitutes
macOS filesystem, wgpu display and model adapters; it does not emulate the
ESP32 CPU or peripherals.
## Runtime lifecycle

Both embedded hosts construct the same `CoreToolHost`. The simulator executes
filesystem and schedule operations against its Mac workspace directory using
the exact ESP constraints. Only `device.status`, `wifi status` and
`reboot` cross a small `PlatformTools` adapter.
The Pi Agent is a first-class, always-resident System App. Its Agent Loop,
context, Tool Registry, and Root View share one Guest and one App lifecycle.
Opening Robinhood or Exa changes only the foreground View; it does not restart
or replace the Agent. Model and native Tool transport complete asynchronously
and return events to that persistent Guest.

Parity is contract-level, not peripheral emulation. The simulator must support
the real embedded Pi Agent, core tool registry, workspace flows and schedules.
It may use macOS storage, networking, deterministic fixtures and simplified
telemetry to do so. CPU load, LittleFS capacity, Wi-Fi/NVS behavior, touch, LCD
scanout and other ESP-IDF details are implemented and accepted only on physical
hardware.
Ordinary Apps receive capability-scoped data roots. Pi Agent alone owns the
top-level `/workspace` and cross-App Tool Registry.

The full macOS host does not link the device UI. Embedded products enable Chat,
Files and Settings. External applications such as Exa or Robinhood belong in
separate plugin/tool and UI adapter crates; Pocket Pi core contains none of
their domain models, clients, credentials or tools.

Settings follows the same host boundary as the rest of the device UI. PocketJS
emits `SettingsCommand` values and renders `SettingsProjection`; only the ESP
host calls ESP-IDF Wi-Fi/NVS/restart APIs. The simulator handles the same
commands with deterministic hardware projections. Password input is transient,
masked, cleared after submit, and never enters the Agent workspace or context.

The physical model boundary has two implementations:

- `UartBackend` sends framed model decisions to the Mac bridge, which can use a
logged-in Codex or Claude Code CLI.
- `WirelessBackend` sends direct HTTPS requests over board Wi-Fi to OpenAI,
OpenRouter, or Anthropic.

Provider JSON and streaming decoders live in `pocket-pi-protocols`; ESP-IDF and
desktop HTTP transports stay in their hosts.

The UART development path is deliberately layered. The ESP firmware owns only
line framing and `UartBackend`; `tools/uart-model-bridge.py` routes those frames;
`tools/uart_bridge` adapts a logged-in Codex app-server or Claude Code stream.
Only decoded top-level `text` is forwarded as UI deltas, while tool-decision
JSON remains private to the Pi Harness. Because ESP logs share UART0, the
firmware disables ESP-IDF logging after the Pi Harness ready handshake so logs
cannot corrupt model frames.

Device time follows the same dual-adapter rule. A standalone Wi-Fi device uses
SNTP. The UART development bridge may seed Unix time at boot, after which the
same persistent `schedule.*` tools and wake loop run unchanged.

## Runtime separation

Agent work runs on a worker thread. UI and touch remain responsive while model
deltas are projected into `ChatProjection`. The UI never owns network access,
model credentials or broker credentials.
## Repository map

## Build entry point
```text
apps/ PocketJS System/ordinary App sources and bundles
crates/pocket-pi-agentos/ App Supervisor and AgentOS contracts
crates/pocket-pi-app-pack/ build-selected embedded App composition
crates/pocket-pi-embedded/ embedded pi-agent-core bridge and host traits
crates/pocket-pi-tools/ native workspace/shell/time/schedule Tools
crates/pocket-pi-protocols/ provider codecs
hosts/esp32-p4-sim/ macOS development simulator for ESP32-P4 contracts
firmware/esp32-p4/ first supported target and reference implementation
tools/uart_bridge/ Mac Codex/Claude streaming adapters
tools/uart-model-bridge.py UART framing and provisioning CLI
```

`cargo xtask` is the orchestration layer:
## Build entry points

```sh
cargo xtask build macos
cargo xtask build agentos-apps
cargo xtask build esp32-p4
cargo xtask build esp32-p4-sim
cargo xtask run esp32-p4-sim
cargo xtask snapshot esp32-p4-sim
```

Simulator proof, firmware compilation, and physical-board proof are separate
evidence tiers. ESP32-P4 is the current reference hardware and physical-board
proof remains its final acceptance tier. Future device targets must provide
their own native composition and physical validation without moving product
logic into firmware.
Loading
Loading