SimBoard

SimBoard is a platform for managing and comparing Earth system simulation metadata, with a focus on E3SM (Energy Exascale Earth System Model) reference simulations.

SimBoard helps researchers:

Store and organize simulation metadata
Browse and visualize simulation details
Compare runs side-by-side
Surface diagnostics and metadata-driven insights

AI-assisted capabilities are being explored to supplement these features, including automated metadata summarization, simulation comparison analysis, and intelligent discovery of patterns across runs.

Prerequisites

Install Docker Desktop
- Download and install: https://www.docker.com/products/docker-desktop
- Ensure Docker Desktop is running before using any Docker-based commands.

Install uv (Python dependency manager)

macOS/Linux:

curl -LsSf https://astral.sh/uv/install.sh | sh

Windows:

powershell -c "irm https://astral.sh/uv/install.ps1 | iex"

Verify installation:
```
uv --version
```

Install Node.js, npm, and pnpm
- Install Node.js (LTS recommended): https://nodejs.org
- Verify Node and npm:
```
node --version
npm --version
```
- Install pnpm:
```
npm install -g pnpm
pnpm --version
```

Clone the repository

git clone https://github.com/E3SM-Project/simboard.git
cd simboard

Developer Quickstart

⚠️ This is a bare-metal development environment and is not production-accurate — it is optimized for speed.

This is the recommended daily workflow:

Fastest hot reloads
Best debugging experience
No Docker overhead

1. Commands

cd simboard

# 1. Setup development assets (env files + certs + DB + deps)
make setup-local

# 2. (First time only) Create an admin account (interactive)
make backend-create-admin

# 3. Start backend (terminal 1)
make backend-run

# 4. Start frontend (terminal 2)
make frontend-run

# 5. Open API and UI
open https://127.0.0.1:8000/docs
open https://127.0.0.1:5173

For a list of all make commands, run:

make help

2. Setup GitHub OAuth Authentication

1. Create a GitHub OAuth App

Go to: https://github.com/settings/developers
Click “New OAuth App”.
Fill in:
- Application name: SimBoard (local)
- Homepage URL: https://127.0.0.1:5173
- Authorization callback URL:
```
https://127.0.0.1:8000/api/v1/auth/github/callback
```
Click Register application.
Copy the generated:
- Client ID
- Client Secret

2. Configure local environment variables

Add the credentials to:

.envs/local/backend.env

Example:

GITHUB_CLIENT_ID=your_client_id
GITHUB_CLIENT_SECRET=your_client_secret

# Must match GitHub OAuth callback URL exactly
GITHUB_REDIRECT_URL=https://127.0.0.1:8000/auth/github/callback

# Generate securely:
# python -c "import secrets; print(secrets.token_urlsafe(64))"
GITHUB_STATE_SECRET_KEY=your_secret

Restart the backend after updating environment variables.

3. Local HTTPS

For local development, SimBoard uses local HTTPS with development certificates:

certs/local.crt
certs/local.key

These files are generated automatically with make install. To re-generate, run:

make gen-certs

Used automatically by:

FastAPI (Uvicorn SSL)
Vite (via VITE_SSL_CERT, VITE_SSL_KEY)

Development Notes

Backend dependencies managed using uv
Frontend dependencies managed using pnpm

Use GitHub Issues to report bugs or propose features. Pull requests should include tests + documentation updates.

Repository Structure

simboard/
├── backend/        # FastAPI, SQLAlchemy, Alembic, OAuth, metadata ingestion
├── frontend/       # Vite + React + Tailwind + shadcn
├── .envs/          # Env configs: example/ (templates, committed) + local/ (developer values, ignored)
├── docker-compose.local.yml
├── docker-compose.yml
├── Makefile        # unified monorepo automation
└── certs/          # dev HTTPS certificates

Pre-commit Hooks

This repository uses pre-commit to enforce consistent checks for both the backend (Python) and frontend (TypeScript).

Pre-commit runs automatically on git commit and will block commits if checks fail.

1. Where to Run It

Always run pre-commit from the repository root.

Some hooks (e.g., mypy) rely on config paths like backend/pyproject.toml. Running from a subdirectory can cause incorrect or inconsistent results.

✅ Correct:

pre-commit run --all-files

❌ Incorrect:

cd backend
pre-commit run --all-files

CI also runs pre-commit from the repo root.

2. What It Checks

Backend:

Ruff (lint + format)
mypy (type checking)

Frontend:

ESLint
Prettier

All hooks are configured in the root .pre-commit-config.yaml.

3. Installation

After cloning:

make install

This will:

Create the backend uv environment (if missing)
Install dependencies
Install git hooks

To reinstall hooks only:

make pre-commit-install

4. Run Manually

make pre-commit-run

Or:

uv run pre-commit run --all-files

5. Notes

Python hooks run via uv
Frontend hooks run via pnpm
Required tools (uv, pnpm, node) must be available in your system PATH
Hooks auto-fix most formatting issues; re-stage files and re-commit if needed

6. Skipping Hooks (Not Recommended)

git commit --no-verify

Use only when absolutely necessary.

Staging and Production Environments

Quick Links

Harbor Registry: https://registry.nersc.gov/harbor/projects
Rancher Dashboard: https://rancher2.spin.nersc.gov/dashboard/c/c-fwj56/explorer/apps.deployment
Full Deployment Guide: docs/cicd/DEPLOYMENT.md

Deployment & CI/CD

SimBoard uses automated CI/CD pipelines to build and deploy containers to NERSC Spin.

Automated Builds (GitHub Actions)

Development Backend: Automatically built and pushed to NERSC registry on every push to main
Development Frontend: Automatically built and pushed to NERSC registry on every push to main
Production Backend & Frontend: Automatically built and pushed on GitHub Releases or version tags (e.g., v0.3.0)

Registry: registry.nersc.gov/e3sm/simboard/

Manual Builds (for testing)

If you need to manually build and push images:

Login to registry:

docker login registry.nersc.gov

Backend:

cd backend
docker buildx build \
  --platform=linux/amd64,linux/arm64 \
  --build-arg ENV=production \
  -t registry.nersc.gov/e3sm/simboard/backend:manual \
  --push \
  .

Frontend:

cd frontend
docker buildx build \
  --platform=linux/amd64,linux/arm64 \
  --build-arg VITE_API_BASE_URL=https://simboard-api.e3sm.org \
  -t registry.nersc.gov/e3sm/simboard/frontend:manual \
  --push \
  .

Provisioning a Service Account for HPC (NERSC Spin)

Service accounts are required when non-interactive systems (e.g., HPC ingestion jobs or automation) need to authenticate to the SimBoard API.

Steps

Ensure the backend is deployed and reachable (e.g., https://simboard-dev-api.e3sm.org).

Run the provisioning script from your local machine:

cd backend

uv run python -m app.scripts.users.provision_service_account \
  --service-name perlmutter-ingestion \
  --base-url https://simboard-dev-api.e3sm.org \
  --admin-email admin@simboard.org \
  --expires-in-days 365

Enter the admin password when prompted.
Copy the generated API token and store it securely (e.g., Kubernetes Secret).

Store as Kubernetes Secret

kubectl -n <namespace> create secret generic simboard-perlmutter-ingestion \
  --from-literal=api_token='<TOKEN>'

Use this secret in your HPC job or service configuration.

> The token is shown only once. Store it securely and rotate as needed.

Example: Ingest Using Service Account Token

Export the token:

export SIMBOARD_API_TOKEN=<TOKEN>

Submit an ingestion request:

curl -X POST https://simboard-dev-api.e3sm.org/api/v1/ingestions/from-path \
  -H "Authorization: Bearer $SIMBOARD_API_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
        "archive_path": "/global/cfs/cdirs/e3sm/simulations/archive.tar.gz",
        "machine_name": "perlmutter",
        "hpc_username": "<your_hpc_username>"
      }'

Python example:

import requests

API_BASE = "https://simboard-dev-api.e3sm.org/api/v1"
TOKEN = "<TOKEN>"

resp = requests.post(
  f"{API_BASE}/ingestions/from-path",
  json={
    "archive_path": "/global/cfs/cdirs/e3sm/simulations/archive.tar.gz",
    "machine_name": "perlmutter",
    "hpc_username": "<your_hpc_username>"
  },
  headers={"Authorization": f"Bearer {TOKEN}"},
)

print(resp.json())

Helpful Docker Commands

docker container ls      # List running containers
docker image ls          # List local images
docker tag <src> <dest>  # Tag an image

For complete deployment instructions, release process, and troubleshooting, see docs/cicd/DEPLOYMENT.md

License

TBD

Name		Name	Last commit message	Last commit date
Latest commit History 41 Commits
.claude		.claude
.envs		.envs
.github		.github
.vite/deps		.vite/deps
.vscode		.vscode
backend		backend
certs		certs
docs		docs
frontend		frontend
.dockerignore		.dockerignore
.gitignore		.gitignore
.pre-commit-config.yaml		.pre-commit-config.yaml
AGENTS.md		AGENTS.md
Makefile		Makefile
README.md		README.md
docker-compose.local.yml		docker-compose.local.yml
docker-compose.yml		docker-compose.yml

E3SM-Project/simboard

Folders and files

Latest commit

History

Repository files navigation

SimBoard

Table of Contents

Prerequisites

Developer Quickstart

1. Commands

2. Setup GitHub OAuth Authentication

1. Create a GitHub OAuth App

2. Configure local environment variables

3. Local HTTPS

Development Notes

Repository Structure

Pre-commit Hooks

1. Where to Run It

2. What It Checks

3. Installation

4. Run Manually

5. Notes

6. Skipping Hooks (Not Recommended)

Staging and Production Environments

Quick Links

Deployment & CI/CD

Automated Builds (GitHub Actions)

Manual Builds (for testing)

Provisioning a Service Account for HPC (NERSC Spin)

Steps

Store as Kubernetes Secret

Example: Ingest Using Service Account Token

Helpful Docker Commands

License

About

Resources

Uh oh!

Stars

Watchers

Forks

Releases

Packages 0

Uh oh!

Contributors

Uh oh!

Languages

Packages