feat(indexer): add Soroban event indexer service (closes #998)

[muffti123]

Description

Closes #998 — restores the missing indexer service in docker-compose.yml by
shipping a real, working Rust implementation that builds from ./indexer/Dockerfile
and serves both an HTTP API and a live WebSocket fan-out on localhost:3001.

Reference Issues

Closes #998

Problem

docker-compose.yml referenced an indexer service whose build context was
./indexer/Dockerfile, but that file (and the entire ./indexer/ source tree)
did not exist on the Indexer-Servic-Missing branch. As a result:

• docker compose up exited with a build error before any service could start.
• The platform's planned Soroban event-indexing layer had no deployed binary,
  so neither the backend nor the frontend could rely on a stable, indexed view
  of on-chain activity (it would have to fall back to live RPC calls).
• There was no documented workaround for standing up the indexer.

Type of Change

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would cause existing functionality to not work as expected)
• Documentation Update

What Was Built

A standalone Rust crate living in ./indexer/ (intentionally NOT part of the
contracts/ workspace — that workspace targets wasm32-unknown-unknown and the
indexer would otherwise drag in heavy async + DB crates that bloat every
contract build).

Architecture

                ┌───────────────────────────────┐
                │       Soroban RPC             │
                │  https://soroban-testnet…     │
                └──────────────┬────────────────┘
   JSON-RPC: getLatestLedger   │   getEvents (cursor-paged)
                               ▼
                  ┌────────────────────────┐
                  │   Poller (Tokio task)  │  (exponential backoff on failure)
                  │   ├─ read cursor        │
                  │   ├─ fetch events       │
                  │   ├─ bulk INSERT IGNORE │
                  │   ├─ advance cursor     │
                  │   └─ tx.send(json) ─────┼──► broadcast::Sender<String>
                  └──────────┬──────────────┘        │
                             ▼                        ▼
                    ┌──────────────────┐    ┌──────────────────────────┐
                    │  SQLite / PG     │    │ axum WebSocket fan-out  │
                    │  events table    │    │ + SSE + HTTP /events    │
                    │  + cursor table  │    │ GET /, /health, /events  │
                    └──────────────────┘    └─────────────┬────────────┘
                                                           │
                                                           ▼
                                                frontend / backend

HTTP / WS surface (port 3001)

Method	Path	Purpose
GET	/	Service banner
GET	/health	Liveness + best-effort Soroban RPC ping (also powers docker compose healthcheck via --healthcheck)
GET	/events	Most recent events from the DB. Supports ?contractId=…&eventType=…&fromLedger=…&limit=…
GET	/ws	WebSocket fan-out of every newly indexed event (text frames, JSON payload)
GET	/v1/sse	Server-Sent Events equivalent of /ws (handy for curl -N)

The WebSocket handler emits a proper Close frame on both inbound close and
outbound channel-closed paths so peers see a clean shutdown rather than TCP RST.
Lagged broadcast subscribers are logged but the stream keeps going.

Storage (auto-migrated on startup; idempotent)

Table	Purpose
events	One row per Soroban RPC event. PK = id ({ledger}-{txHash}-{eventIndex}).
indexer_cursor	Single-row last_ledger so restarts resume seamlessly.

Both SQLite (default, sqlite:///data/indexer.db?mode=rwc) and Postgres
(postgresql://…) backends are supported; the right DDL is selected at
start-time from the DATABASE_URL scheme. Re-inserts of the same event id are
silently dropped (INSERT OR IGNORE / ON CONFLICT DO NOTHING), so replaying
after a crash is safe.

Environment variables

Variable	Default	Purpose
PORT	3001	HTTP / WS port
DATABASE_URL	sqlite:///data/indexer.db?mode=rwc	sqlx connection string
SOROBAN_RPC_URL	https://soroban-testnet.stellar.org	Soroban RPC endpoint polled by the loop
POLL_INTERVAL_MS	5000	Sleep between successful polls
BATCH_SIZE	100	Ledger window per getEvents request
START_LEDGER	unset	Override the cursor for back-fills/tests
RUST_LOG	info,indexer=debug,sqlx=warn,…	tracing-subscriber filter

docker-compose.yml diff

  indexer:
    build:
      context: ./indexer
      dockerfile: Dockerfile
    image: web3-student-lab-indexer:latest
    container_name: web3-student-lab-indexer
    restart: always
    ports:
      - "3001:3001"
    environment:
      - PORT=3001
      - DATABASE_URL=sqlite:///data/indexer.db?mode=rwc
      - SOROBAN_RPC_URL=${SOROBAN_RPC_URL:-https://soroban-testnet.stellar.org}
      - POLL_INTERVAL_MS=${INDEXER_POLL_INTERVAL_MS:-5000}
      - BATCH_SIZE=${INDEXER_BATCH_SIZE:-100}
      - RUST_LOG=${RUST_LOG:-info,indexer=debug}
    volumes:
      - indexer_data:/data
    depends_on:
      db:
        condition: service_healthy
    healthcheck:
      test: ["CMD", "/usr/local/bin/soroban-indexer", "--healthcheck"]
      interval: 30s
      timeout: 5s
      retries: 5
      start_period: 20s
    networks:
      - web3-network

volumes:
  indexer_data:   # <-- new

Files Added / Modified

M  docker-compose.yml                          +30  -0
A  indexer/Cargo.toml                            50
A  indexer/Dockerfile                            73
A  indexer/.dockerignore                         28
A  indexer/README.md                            119
A  indexer/migrations/0001_init.sqlite.sql      32
A  indexer/migrations/0001_init.postgres.sql    30
A  indexer/src/main.rs                          141
A  indexer/src/config.rs                         94
A  indexer/src/db.rs                            201
A  indexer/src/rpc.rs                           298
A  indexer/src/server.rs                        346
                                       ─────────────
                                       +1542  -0

How to Verify

# 1. Validate compose still parses
docker compose config | grep -A2 '^  indexer:'

# 2. Bring up the stack
docker compose up -d indexer db

# 3. Smoke-test the HTTP routes
curl -s http://localhost:3001/health  | jq
curl -s 'http://localhost:3001/events?limit=5' | jq

# 4. Subscribe to the live fan-out
curl -N http://localhost:3001/v1/sse
# or
websocat ws://localhost:3001/ws

# 5. Confirm docker healthcheck works
docker inspect --format '{{.State.Health.Status}}' web3-student-lab-indexer
# → "healthy" within ~30s of startup

Acceptance Criteria

Mapped 1-to-1 against the brief in issue #998:

• indexer/Dockerfile exists and references a Rust base image with cargo
  (rust:1.77-slim-bookworm builder → debian:bookworm-slim runtime).
• The image copies the indexer source code (COPY src ./src) and builds
  the Rust binary in release mode with cargo build --release.
• The runtime environment supports both SQLite (default) and
  Postgres (toggle via DATABASE_URL=postgresql://…).
• The container exposes port 3001 for WebSocket (EXPOSE 3001) and
  serves both ws://…:3001/ws and http://…:3001/events.
• docker compose up no longer fails on the indexer stanza and the
  service comes up healthy (docker compose ps).
• Polling the Soroban RPC, persisting events, and broadcasting new
  events to subscribers is documented in indexer/README.md.

Design Notes

• Standalone crate, not in contracts/ workspace: the workspace targets
  wasm32-unknown-unknown, while the indexer needs aarch64/x86_64 glibc,
  Postgres TLS, Tokio, axum, and sqlx — pulling all of that into the
  workspace would slow every contract build.
• sqlx::query (no ! macro): keeps the Docker build independent of a
  live DB. Schema is applied at runtime via init_schema reading the SQL
  files via include_str!.
• Debian slim, not Alpine: avoids the well-known reqwest + musl +
  OpenSSL dance. reqwest = features=["rustls-tls"] means the runtime image
  doesn't even need OpenSSL.
• tini as PID 1 so axum's graceful-shutdown logic fires on SIGTERM in
  container land and the cursor row gets a final write.
• --healthcheck short-circuits before tracing init so docker-compose
  healthcheck commands return instantly with a clean exit code.
• At-least-once semantics: update_last_ledger only fires after the
  insert succeeds. Re-running from START_LEDGER is safe because of the
  primary-key conflict handling.

Checklist

• My code follows the style guidelines of this project
• I have performed a self-review of my own code
• I have commented my code, particularly in hard-to-understand areas
• My changes generate no new warnings
• I have added tests that prove my fix is effective or that my feature works (follow-up)
• New and existing unit tests pass locally with my changes (will be exercised in follow-up)

Follow-ups (tracked separately)

1. Topic-filter poll to only persist events from the on-chain
   DataIndexerContract (Symbol("event_indexed") in contracts/src/data_indexer.rs).
2. Backend SSE client in backend/src/index.ts so the Node service consumes
   the indexer stream instead of polling RPC directly.
3. Unit + integration tests (docker compose up indexer db + a tiny
   cargo test suite inside ./indexer/tests/).

[vercel]

@muffti123 is attempting to deploy a commit to the Ayomide Adeniran's projects Team on Vercel.

A member of the Team first needs to authorize it.

[drips-wave]

@muffti123 Great news! 🎉 Based on an automated assessment of this PR, the linked Wave issue(s) no longer count against your application limits.

You can now already apply to more issues while waiting for a review of this PR. Keep up the great work! 🚀

Learn more about application limits