feat(api): optional FastAPI service layer (apis/)#111
Open
Sammyjoseph999 wants to merge 17 commits into
Open
Conversation
Port the HTTP service into this repo, reconciled to the current package: - imports use fully-qualified climate_tookit.* paths (dropped the fragile sys.path hacks and 'from sources' / 'from source_data' shortcuts) - ensemble call sites updated to the current 'model_workers' kwarg (was 'max_workers') across hazards/statistics/compare/climatology - fastapi/uvicorn/jinja2/python-multipart added as an optional 'api' extra so the base CLI/library stays lean - README documents install + run (uvicorn apis.main:app) Verified the app boots and all routers register (/, /docs, /openapi.json, /api/v1/data/sources return 200); ruff clean. GEE-backed endpoints require live Earth Engine auth to exercise end to end.
Fixes reported UI issues:
- source dropdowns were hardcoded per-template with stale/invalid keys
('chirps', 'hirts', 'chirps+chirts') and incomplete lists. Centralized a
single valid source catalog (apis/catalog.py) exposed to all templates via
Jinja globals; every module now offers the correct current source keys.
- climatology defaulted to the invalid 'chirps' source, so it always failed;
it now defaults to agera_5 with a valid list.
- ensembles were only reachable via the JSON API. Added NEX-GDDP ensemble
controls (scenario + model multi-select) to the climatology page, wired to
calculate_climatology_ensemble when source=nex_gddp.
Verified all UI pages render 200 and the ensemble controls appear; ruff clean.
…/periods Source catalog now reflects what each module actually supports: - analysis modules (statistics/seasons/hazards/compare_periods) default to 'auto' (recommended: CHIRPS v3 + AgERA5 with fallbacks) plus standalone daily temp+precip sources (agera_5/era_5/nasa_power) and nex_gddp; dropped monthly TerraClimate from these daily modules. 'paired' is deferred until the forms collect precip/temp sources. - climatology/compare-datasets keep concrete single-source lists. Ensemble UI (previously only on climatology) now also on statistics, hazards and compare-periods via shared _ensemble_controls / _ensemble_result partials: choosing NEX-GDDP reveals scenario + model selectors and routes to the matching ensemble function (analyze_ensemble_nex_gddp / calculate_ensemble / ensemble_compare). All 8 UI pages render 200; ensemble controls on all 4 modules; ruff clean.
… JSON) Surface scenario / models-used / failed-model counts above the payload when present (all guarded, so it degrades to just the JSON for any shape), and move the full result into a collapsible <details>. Climatology now uses the shared _ensemble_result partial too. All UI pages render 200; ruff clean.
Adds a shared client-side download button (_download_button.html) to the result view of every module (fetch, statistics, seasons, climatology, hazards, compare-periods, compare-datasets). Serializes the already-computed result to a JSON file with no server round-trip or recompute. Verified the button and embedded payload render for a mock success result; all pages 200, ruff clean.
calculate_climatology already writes annual/monthly PNGs when given an output_dir; the climatology handler now points it at a served artifacts directory (outputs/api, git-ignored) mounted at /artifacts, forces the Agg matplotlib backend for server-thread safety, and passes the resulting plot URLs to the template, which renders them as images. Verified: artifacts route serves files (200), path->URL mapping correct, template renders <img>. Actual plot content requires a live GEE session. Downloads (JSON) already cover all modules; visualizations start with climatology as the exemplar.
Point compare_sources/print_report at a unique served artifacts subdir per run (outputs/api/compare_datasets/<id>, git-ignored), glob the generated PNGs and render them as images on the results page. Verified pages render 200 and the <img> block renders for mock plot URLs; ruff clean. Plot content needs a live GEE session.
Handlers parsed form fields (float/int) outside their try/except, so a missing or invalid field raised an uncaught exception and the browser showed a bare 'Internal Server Error' on every module. Add an app-level exception handler that renders the actual exception type/message (+ traceback) as a readable page for HTML routes and JSON for /api routes, so failures are diagnosable rather than opaque. Verified a previously-uncaught error now renders a 'Server error' page with the real exception; ruff clean.
Two robustness fixes for the reported 'Internal Server Error on every module': - Startup guard: if python-multipart is not installed (the most likely cause of a universal form-POST 500, since request.form() fails before any handler runs), refuse to start with a clear 'uv sync --extra api' message instead of an opaque per-request 500. - Moved form parsing (await request.form() + payload float/int conversions) inside each page handler's try/except, so a missing/invalid field now renders a clean inline error on that module's page instead of escaping as a 500. Verified: all pages render 200; a missing-field POST now shows an inline module error (not the global error page); ruff clean.
Member
|
I haven't had the chance to do a full review of this yet, but initial thought is that it might be helpful to have the api dockerized with an example of how to deploy it via docker. Awesome work on this though, I'm excited to test it and see how it goes! |
The download partial now offers 'Download (CSV)' next to '(JSON)'. Client-side and generic: when the result has a season_statistics list it exports a proper per-season table; otherwise it flattens the nested result to a field,value CSV. Works for every module with no server round-trip. Verified the partial renders both buttons/handlers and all pages compile 200.
4 tasks
Rename climate_tookit -> climate_toolkit across apis/ to match the merged package rename (#116); resolve uv.lock and regenerate with the api extra. Verified: app imports, all pages render 200, ruff clean, 22 packaging/cli tests pass.
Collaborator
Author
|
Thanks @bjyberg! Added Docker support for the service:
Verified locally: image builds with no compiler, container boots and returns 200 on docker build -f Dockerfile.api -t climate-toolkit-api .
docker run --rm -p 8000:8000 climate-toolkit-api # then open http://127.0.0.1:8000/ |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds an optional FastAPI service layer (
apis/) that exposes the toolkit over HTTP — a JSON API under/api/v1/*plus simple HTML pages — reconciled to the current codebase.What's included
apis/: FastAPI app, 7 routers (data, statistics, hazards, compare, seasons, climatology, compare-datasets), Pydantic schemas, Jinja templates, static assets.api(fastapi/uvicorn/jinja2/python-multipart) so the base CLI/library stays lean:Reconciliation to current code
climate_tookit.*paths (removed fragilesys.pathhacks andfrom sources/from source_datashortcuts).model_workerskwarg (wasmax_workers).apis/catalog.py): analysis modules default toauto(CHIRPS v3 + AgERA5 with fallbacks) plus standalone daily temp+precip sources and NEX-GDDP; removed invalid keys (chirps,hirts,chirps+chirts) and monthly TerraClimate from the daily modules.Verification
/docs//openapi.jsonreturn 200; ensemble controls render on all four modules;ruffclean.