Skip to content

[feat] Implementation of Http Handshake RTC Audio and File Transfer Pipeline - #388

Merged
imxade merged 4 commits into
AOSSIE-Org:mainfrom
PinJinx:HTTPHandshake
Aug 24, 2026
Merged

[feat] Implementation of Http Handshake RTC Audio and File Transfer Pipeline#388
imxade merged 4 commits into
AOSSIE-Org:mainfrom
PinJinx:HTTPHandshake

Conversation

@PinJinx

@PinJinx PinJinx commented Aug 24, 2026

Copy link
Copy Markdown
Contributor

Addressed Issues:

Fixes #171 #375 #377

Description

This PR addresses the issues reported in #171, #375, and #377 by improving the handshake siginalling between the client and the server while also adding in FTP based File Transfer system between the server and all the clients.

Screenshots/Recordings:

Files.mp4

Functional Verification

  • Please check off the behaviors verified with this change.

Screen Mirror

  • Screen MIrror works.

Authentication

  • Connection doesn't work without a valid token.

Basic Gestures

  • One-finger tap: Verified as Left Click.

  • Two-finger tap: Verified as Right Click.

  • Click and drag: Verified selection behavior.

  • Pinch to zoom: Verified zoom functionality (if applicable).

Modes & Settings

  • Cursor mode: Cursor moves smoothly and accurately.

  • Scroll mode: Page scrolls as expected.

  • Sensitivity: Verified changes in cursor speed/sensitivity settings.

  • Copy and Paste: Verified both Copy and Paste functionality.

  • Invert Scrolling: Verified scroll direction toggles correctly.

Advanced Input

  • Key combinations: Verified "hold" behavior for modifiers (e.g., Ctrl+C) and held keys are shown in buffer.

  • Keyboard input: Verified Space, Backspace, and Enter keys work correctly.

  • Glide typing: Verified path drawing and text output.

  • Voice input: Verified speech-to-text functionality for full sentences.

  • Backspace doesn't send the previous input.

Any other gesture or input behavior introduced:

  • New Gestures: Verified any other gesture or input behavior introduced in this PR.

Checklist

  • My PR addresses a single issue, fixes a single bug or makes a single improvement.

  • My code follows the project's code style and conventions

  • I have performed a self-review of my own code

  • I have commented my code, particularly in hard-to-understand areas

  • If applicable, I have made corresponding changes or additions to the documentation

  • If applicable, I have made corresponding changes or additions to tests

  • My changes generate no new warnings or errors

  • I have joined the and I will share a link to this PR with the project maintainers there

  • I have read the

  • Once I submit my PR, CodeRabbit AI will automatically review it and I will address CodeRabbit's comments.

  • Incase of UI change I've added a demo video.

⚠️ AI Notice - Important!
We encourage contributors to use AI tools responsibly when creating Pull Requests. While AI can be a valuable aid, it is essential to ensure that your contributions meet the task requirements, build successfully, include relevant tests, and pass all linters. Submissions that do not meet these standards may be closed without warning to maintain the quality and integrity of the project. Please take the time to understand the changes you are proposing and their impact.

Summary by CodeRabbit

  • New Features
    • Added file sharing with drag-and-drop uploads, progress tracking, downloads, deletion, and incoming-file notifications.
    • Added browser-based WebRTC session signaling with improved connection and recovery handling.
    • Added audio streaming alongside video, with configurable frame rate and audio source.
    • Added authenticated debug status, latency reporting, and live log viewing.
  • Bug Fixes
    • Improved screen mirroring playback, autoplay fallback, unmuting, and stream cleanup.
  • Configuration
    • Enabled 60 FPS video and system audio processing by default.

PinJinx and others added 4 commits August 24, 2026 11:25
- Bump typescript from ^5.7.2 to ^7.0.0
- TS 7.0 is a native Go port (Project Corsa), GA July 8 2026;
  type-checking semantics are unchanged from 6.0
- Verified: tsc --noEmit passes with zero errors, npm run build
  (Vite + Nitro) succeeds
- Biome lint errors (56, pre-existing) and missing test files were
  confirmed unrelated to this change via git stash comparison
  against the prior TS version
- No programmatic-compiler-API dependents in this stack (Vite uses
  bundler mode with noEmit:true; Biome is a separate Rust toolchain),
  so the 7.1 API gap does not apply here

Closes AOSSIE-Org#381
Delete node_modules and package-lock.json and reinstall fresh, per
maintainer request (@PinJinx), to resolve the Windows CI failure
where npm ci reported missing lru-cache/typescript entries.

Also picks up transitive dependency drift in the Oxc/Rolldown
toolchain used by Vite (unrelated to TypeScript itself). Verified
tsc --noEmit and npm run build both still pass cleanly.
@PinJinx
PinJinx requested a review from imxade August 24, 2026 14:10
@coderabbitai

coderabbitai Bot commented Aug 24, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

Caution

Review failed

Failed to post review comments.

We encountered an issue with GitHub. Use @coderabbitai full review to retry the review.

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: 7ad3e01d-7f42-4cc1-8b44-eba8db5266b3

📥 Commits

Reviewing files that changed from the base of the PR and between 37d898b and 4cebc7b.

⛔ Files ignored due to path filters (1)
  • package-lock.json is excluded by !**/package-lock.json
📒 Files selected for processing (23)
  • package.json
  • src/components/FileTransfer/IncomingFileNotification.tsx
  • src/components/FileTransfer/SendFileComponent.tsx
  • src/components/Trackpad/ScreenMirror.tsx
  • src/contexts/FileShareContext.tsx
  • src/hooks/useWebRtcStream.ts
  • src/routes/__root.tsx
  • src/routes/debug.tsx
  • src/server-config.json
  • src/server/constants.ts
  • src/server/gstreamer/gstManager.ts
  • src/server/nitro-plugin.ts
  • src/server/siginalling/handlers/debug.ts
  • src/server/siginalling/handlers/fileShare.ts
  • src/server/siginalling/handlers/rtc.ts
  • src/server/siginalling/handlers/sessionStore.ts
  • src/server/siginalling/handlers/udpSocket.ts
  • src/server/siginalling/server.ts
  • src/server/siginalling/utils.ts
  • src/server/siginalling/webRTC.ts
  • src/server/webRTC.ts
  • src/utils/configHelper.ts
  • vite.config.ts
💤 Files with no reviewable changes (1)
  • src/server/webRTC.ts

Included review availability: Your plan provides up to 1 included review per hour; 0 remain after this review.

⏰ Context from checks skipped due to timeout. (3)
  • GitHub Check: build-electron (ubuntu-latest)
  • GitHub Check: build-electron (windows-latest)
  • GitHub Check: build-electron (macos-latest)
🧰 Additional context used
📓 Path-based instructions (7)
**/*

📄 CodeRabbit inference engine (AGENTS.md)

A change is not finished until npm run check and npm run test pass with zero errors locally.

Files:

  • src/server-config.json
  • package.json
  • vite.config.ts
  • src/server/constants.ts
  • src/routes/__root.tsx
  • src/server/nitro-plugin.ts
  • src/routes/debug.tsx
  • src/server/siginalling/utils.ts
  • src/components/Trackpad/ScreenMirror.tsx
  • src/components/FileTransfer/SendFileComponent.tsx
  • src/server/siginalling/handlers/udpSocket.ts
  • src/server/siginalling/handlers/fileShare.ts
  • src/components/FileTransfer/IncomingFileNotification.tsx
  • src/server/siginalling/handlers/sessionStore.ts
  • src/utils/configHelper.ts
  • src/contexts/FileShareContext.tsx
  • src/server/gstreamer/gstManager.ts
  • src/server/siginalling/handlers/rtc.ts
  • src/server/siginalling/webRTC.ts
  • src/hooks/useWebRtcStream.ts
  • src/server/siginalling/handlers/debug.ts
  • src/server/siginalling/server.ts
**/*.{ts,tsx,cjs}

📄 CodeRabbit inference engine (AGENTS.md)

**/*.{ts,tsx,cjs}: Before writing a new file, read the nearest existing file that does the same job and follow its structure, naming, and idioms.
Use brief comments to explain non-obvious implementation reasons (e.g. FFI struct layouts, SCTP delayedSackTime: 0 tuning, or mobile keyboard sentinel characters). Do not leave commented-out code or multi-paragraph docstrings.

Files:

  • vite.config.ts
  • src/server/constants.ts
  • src/routes/__root.tsx
  • src/server/nitro-plugin.ts
  • src/routes/debug.tsx
  • src/server/siginalling/utils.ts
  • src/components/Trackpad/ScreenMirror.tsx
  • src/components/FileTransfer/SendFileComponent.tsx
  • src/server/siginalling/handlers/udpSocket.ts
  • src/server/siginalling/handlers/fileShare.ts
  • src/components/FileTransfer/IncomingFileNotification.tsx
  • src/server/siginalling/handlers/sessionStore.ts
  • src/utils/configHelper.ts
  • src/contexts/FileShareContext.tsx
  • src/server/gstreamer/gstManager.ts
  • src/server/siginalling/handlers/rtc.ts
  • src/server/siginalling/webRTC.ts
  • src/hooks/useWebRtcStream.ts
  • src/server/siginalling/handlers/debug.ts
  • src/server/siginalling/server.ts
**/*.{ts,tsx}

📄 CodeRabbit inference engine (AGENTS.md)

Do not introduce any types, and do not use as to suppress real type errors.

Files:

  • vite.config.ts
  • src/server/constants.ts
  • src/routes/__root.tsx
  • src/server/nitro-plugin.ts
  • src/routes/debug.tsx
  • src/server/siginalling/utils.ts
  • src/components/Trackpad/ScreenMirror.tsx
  • src/components/FileTransfer/SendFileComponent.tsx
  • src/server/siginalling/handlers/udpSocket.ts
  • src/server/siginalling/handlers/fileShare.ts
  • src/components/FileTransfer/IncomingFileNotification.tsx
  • src/server/siginalling/handlers/sessionStore.ts
  • src/utils/configHelper.ts
  • src/contexts/FileShareContext.tsx
  • src/server/gstreamer/gstManager.ts
  • src/server/siginalling/handlers/rtc.ts
  • src/server/siginalling/webRTC.ts
  • src/hooks/useWebRtcStream.ts
  • src/server/siginalling/handlers/debug.ts
  • src/server/siginalling/server.ts
**/*.{ts,tsx,js,jsx}

⚙️ CodeRabbit configuration file

**/*.{ts,tsx,js,jsx}: NextJS:

  • Ensure that "use client" is being used
  • Ensure that only features that allow pure client-side rendering are used
  • NextJS best practices (including file structure, API routes, and static generation methods) are used.

TypeScript:

  • Avoid 'any', use explicit types
  • Prefer 'import type' for type imports
  • Review for significant deviations from Google JavaScript style guide. Minor style issues are not a priority
  • The code adheres to best practices associated with React
  • The code adheres to best practices associated with React PWA
  • The code adheres to best practices associated with SPA
  • The code adheres to best practices recommended by lighthouse or similar tools for performance
  • The code adheres to best practices associated with Node.js
  • The code adheres to best practices recommended for performance

Security:

  • No exposed API keys or sensitive data
  • Use expo-secure-store for sensitive storage
  • Validate deep linking configurations
  • Check for common security vulnerabilities such as:
    • SQL Injection
    • XSS (Cross-Site Scripting)
    • CSRF (Cross-Site Request Forgery)
    • Insecure dependencies
    • Sensitive data exposure

Internationalization:

  • User-visible strings should be externalized to resource files (i18n)

Files:

  • vite.config.ts
  • src/server/constants.ts
  • src/routes/__root.tsx
  • src/server/nitro-plugin.ts
  • src/routes/debug.tsx
  • src/server/siginalling/utils.ts
  • src/components/Trackpad/ScreenMirror.tsx
  • src/components/FileTransfer/SendFileComponent.tsx
  • src/server/siginalling/handlers/udpSocket.ts
  • src/server/siginalling/handlers/fileShare.ts
  • src/components/FileTransfer/IncomingFileNotification.tsx
  • src/server/siginalling/handlers/sessionStore.ts
  • src/utils/configHelper.ts
  • src/contexts/FileShareContext.tsx
  • src/server/gstreamer/gstManager.ts
  • src/server/siginalling/handlers/rtc.ts
  • src/server/siginalling/webRTC.ts
  • src/hooks/useWebRtcStream.ts
  • src/server/siginalling/handlers/debug.ts
  • src/server/siginalling/server.ts
src/server/**/*.ts

📄 CodeRabbit inference engine (AGENTS.md)

src/server/**/*.ts: input message sanitization and 8ms throttling belong in InputHandler; platform FFI bindings live in src/server/drivers/.
Never revert to dual peer connections or HTTP SSE signaling: Rein uses werift on the server with a single RTCPeerConnection per viewer session and WebSocket signaling (/ws).
API updates to POST /api/config MUST call WebRTCManager.updateConfig() so active InputHandler instances receive updated sensitivity/scroll inversion without restarting the server.
Never write directly to src/server-config.json in dev mode.
Sensitive administration endpoints (POST /api/auth/token) are restricted to localhost only and return 403 to remote callers.
All token lookups call crypto.timingSafeEqual(). Tokens persist to tokens.json with file permissions 0o600 (owner read/write only).

Files:

  • src/server/constants.ts
  • src/server/nitro-plugin.ts
  • src/server/siginalling/utils.ts
  • src/server/siginalling/handlers/udpSocket.ts
  • src/server/siginalling/handlers/fileShare.ts
  • src/server/siginalling/handlers/sessionStore.ts
  • src/server/gstreamer/gstManager.ts
  • src/server/siginalling/handlers/rtc.ts
  • src/server/siginalling/webRTC.ts
  • src/server/siginalling/handlers/debug.ts
  • src/server/siginalling/server.ts
src/**/*.{ts,tsx}

📄 CodeRabbit inference engine (AGENTS.md)

src/**/*.{ts,tsx}: Touch motion must use input-unordered: move, scroll, zoom, and touch events MUST be sent via input-unordered (maxRetransmits: 0).
The hidden mobile keyboard <input> requires defaultValue=" " (space sentinel) to reliably detect backspaces (deleteContentBackward) on mobile WebViews.

Files:

  • src/server/constants.ts
  • src/routes/__root.tsx
  • src/server/nitro-plugin.ts
  • src/routes/debug.tsx
  • src/server/siginalling/utils.ts
  • src/components/Trackpad/ScreenMirror.tsx
  • src/components/FileTransfer/SendFileComponent.tsx
  • src/server/siginalling/handlers/udpSocket.ts
  • src/server/siginalling/handlers/fileShare.ts
  • src/components/FileTransfer/IncomingFileNotification.tsx
  • src/server/siginalling/handlers/sessionStore.ts
  • src/utils/configHelper.ts
  • src/contexts/FileShareContext.tsx
  • src/server/gstreamer/gstManager.ts
  • src/server/siginalling/handlers/rtc.ts
  • src/server/siginalling/webRTC.ts
  • src/hooks/useWebRtcStream.ts
  • src/server/siginalling/handlers/debug.ts
  • src/server/siginalling/server.ts
src/server/**/*.{ts,json}

📄 CodeRabbit inference engine (AGENTS.md)

Do not modify local UDP relay port 5004: GStreamer outputs RTP to 127.0.0.1:5004 (RTP_HOST, RTP_PORT). Changing this port requires updating src/server/constants.ts and GstManager.

Files:

  • src/server/constants.ts
  • src/server/nitro-plugin.ts
  • src/server/siginalling/utils.ts
  • src/server/siginalling/handlers/udpSocket.ts
  • src/server/siginalling/handlers/fileShare.ts
  • src/server/siginalling/handlers/sessionStore.ts
  • src/server/gstreamer/gstManager.ts
  • src/server/siginalling/handlers/rtc.ts
  • src/server/siginalling/webRTC.ts
  • src/server/siginalling/handlers/debug.ts
  • src/server/siginalling/server.ts
🪛 ast-grep (0.45.1)
src/server/siginalling/handlers/sessionStore.ts

[warning] 118-118: Avoid logging sensitive data
Context: logger.info(Cleaning up session: ${sessionId})
Note: [CWE-532] Insertion of Sensitive Information into Log File.

(log-sensitive-data-typescript)

src/server/gstreamer/gstManager.ts

[warning] Importing child_process exposes a command-execution surface; ensure any command/argument built from input is validated, and prefer execFile/spawn with an argument array over exec.
Context: import { spawn, execSync, type ChildProcess } from "node:child_process"
Note: [CWE-78] Improper Neutralization of Special Elements used in an OS Command ('OS Command Injection').

(detect-child-process-typescript)

src/server/siginalling/webRTC.ts

[warning] 111-113: Avoid logging sensitive data
Context: logger.error(
InputHandler error [${sessionId}]: ${errorType} – ${message},
)
Note: [CWE-532] Insertion of Sensitive Information into Log File.

(log-sensitive-data-typescript)


[warning] 151-151: Avoid logging sensitive data
Context: logger.error(Input handler error [${sessionId}]: ${String(err)})
Note: [CWE-532] Insertion of Sensitive Information into Log File.

(log-sensitive-data-typescript)


[warning] 154-154: Avoid logging sensitive data
Context: logger.error(Input parse error [${sessionId}]: ${String(err)})
Note: [CWE-532] Insertion of Sensitive Information into Log File.

(log-sensitive-data-typescript)


[warning] 193-193: Avoid logging sensitive data
Context: logger.info(ICE state [${sessionId}]: ${state})
Note: [CWE-532] Insertion of Sensitive Information into Log File.

(log-sensitive-data-typescript)


[warning] 202-202: Avoid logging sensitive data
Context: logger.info(Session created, offer sent: ${sessionId})
Note: [CWE-532] Insertion of Sensitive Information into Log File.

(log-sensitive-data-typescript)


[warning] 205-205: Avoid logging sensitive data
Context: logger.error(Failed to create offer [${sessionId}]: ${String(err)})
Note: [CWE-532] Insertion of Sensitive Information into Log File.

(log-sensitive-data-typescript)


[warning] 228-228: Avoid logging sensitive data
Context: logger.info(Answer applied for session: ${sessionId})
Note: [CWE-532] Insertion of Sensitive Information into Log File.

(log-sensitive-data-typescript)


[warning] 231-231: Avoid logging sensitive data
Context: logger.error(Failed to apply answer [${sessionId}]: ${String(err)})
Note: [CWE-532] Insertion of Sensitive Information into Log File.

(log-sensitive-data-typescript)


[warning] 255-255: Avoid logging sensitive data
Context: logger.error(Failed to add ICE candidate [${sessionId}]: ${String(err)})
Note: [CWE-532] Insertion of Sensitive Information into Log File.

(log-sensitive-data-typescript)


[warning] 291-291: Avoid logging sensitive data
Context: logger.info(Session cleanly disconnected: ${sessionId})
Note: [CWE-532] Insertion of Sensitive Information into Log File.

(log-sensitive-data-typescript)

🪛 React Doctor (0.9.3)
src/components/FileTransfer/SendFileComponent.tsx

[warning] 191-191: Blind users can't tell what this control does because screen readers find no label, so add visible text, aria-label, or aria-labelledby.

Give every interactive control a label screen readers can read.

(control-has-associated-label)


[warning] 353-353: Blind users can't tell what this control does because screen readers find no label, so add visible text, aria-label, or aria-labelledby.

Give every interactive control a label screen readers can read.

(control-has-associated-label)


[warning] 362-362: Blind users can't tell what this control does because screen readers find no label, so add visible text, aria-label, or aria-labelledby.

Give every interactive control a label screen readers can read.

(control-has-associated-label)

src/components/FileTransfer/IncomingFileNotification.tsx

[error] 273-273: This ref is mutated during render. React can replay or discard render work, so the mutation can leak from UI that never commits.

Move ref writes into an event handler or effect. Render must stay pure because React can replay or discard it. The predictable null-guarded lazy initialization pattern remains supported.

(no-ref-current-in-render)


[warning] 273-273: This component misses React Compiler's automatic memoization & re-renders more than it should: Cannot access refs during render. Rewrite the flagged code so the compiler can optimize it.

React refs are values that are not needed for rendering. Refs should only be accessed outside of render, such as in event handlers or effects. Accessing a ref value (the current property) during render can cause your component not to update as expected (https://react.dev/reference/react/useRef).

(refs)

Walkthrough

The change adds remote file transfer with authenticated uploads, downloads, deletion, and SSE notifications. It replaces WebSocket WebRTC signaling with HTTP and SSE. It adds audio RTP streaming, configurable video framerate, updated telemetry, and client playback recovery.

Changes

Remote control and file transfer

Layer / File(s) Summary
HTTP utilities and route wiring
src/server/siginalling/utils.ts, src/server/siginalling/server.ts, src/server/siginalling/handlers/debug.ts, src/server/nitro-plugin.ts, vite.config.ts, package.json
Shared request parsing and authentication utilities support the extracted debug handlers, new RTC routes, and file-sharing routes. Signaling imports now use the siginalling path.
File transfer service and client UI
src/server/siginalling/handlers/fileShare.ts, src/contexts/FileShareContext.tsx, src/components/FileTransfer/*, src/routes/__root.tsx
The server stores files in memory and broadcasts file events over SSE. The client supports uploads, progress, downloads, deletion, acceptance, rejection, notifications, and overlay controls.
HTTP WebRTC sessions
src/server/siginalling/handlers/rtc.ts, src/server/siginalling/handlers/sessionStore.ts, src/server/siginalling/handlers/udpSocket.ts, src/server/siginalling/webRTC.ts, src/hooks/useWebRtcStream.ts
WebRTC offer, answer, ICE, session SSE, and deletion flows use authenticated HTTP requests. Session state, event buffering, cleanup, RTP forwarding, retry handling, and multi-track support are implemented.
Media configuration and session telemetry
src/server/gstreamer/gstManager.ts, src/server/constants.ts, src/utils/configHelper.ts, src/server-config.json, src/components/Trackpad/ScreenMirror.tsx, src/routes/debug.tsx
The server adds configurable video framerate and Opus audio RTP output. Screen playback handles autoplay restrictions. Debug views report SSE connection presence instead of viewer counts.

Estimated code review effort: 5 (Critical) | ~120 minutes

Merge Risk: 🟠 High · up to 4cebc

This PR adds HTTP/SSE signaling and in-memory file transfer. At the current head, uploads can consume excessive server memory, unanswered handshakes can leak peer connections and ports, and stalled signaling or UDP bind failures can leave users stuck or the server retrying aggressively. These high-impact availability and correctness risks should be fixed or explicitly accepted before merge.

Suggested labels: Typescript Lang

Suggested reviewers: imxade

Poem

I’m a rabbit with packets tucked under my ear,
Files hop through the network, arriving clear.
Offers and answers now travel in streams,
Audio joins video in GStreamer dreams.
I thump both paws: the new routes are bright!

🚥 Pre-merge checks | ✅ 3 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Out of Scope Changes check ⚠️ Warning The changes include WebRTC, screen-mirror, audio, configuration, and debug updates that are not covered by the provided file-transfer issue #171. Split unrelated changes into separate pull requests or link the relevant issues and document why these changes are required.
✅ Passed checks (3 passed)
Check name Status Explanation
Title check ✅ Passed The title accurately identifies the HTTP handshake work, but it does not mention the file-transfer feature.
Description check ✅ Passed The description includes linked issues, change details, verification results, and a recording; several template checklist items remain incomplete.
Linked Issues check ✅ Passed The implementation provides authenticated upload, listing, download, deletion, and event streaming for remote file transfer requested by issue #171.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@PinJinx PinJinx changed the title Http handshake [feat] Implementation of Http Handshake RTC Audio and File Transfer Pipeline Aug 24, 2026
@imxade
imxade merged commit 628de81 into AOSSIE-Org:main Aug 24, 2026
8 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Feature]: Design and Integrate Remote File Transfer mechanism for Rein

3 participants