Skip to content

feat(storage): wire up Neon Object Storage adapter - #284

Merged
kisernl merged 3 commits into
masterfrom
devin/1722948780-neon-storage
Aug 7, 2026
Merged

feat(storage): wire up Neon Object Storage adapter#284
kisernl merged 3 commits into
masterfrom
devin/1722948780-neon-storage

Conversation

@kisernl

@kisernl kisernl commented Aug 6, 2026

Copy link
Copy Markdown
Collaborator

Wires the new @storagesdk/adapters/neon adapter into the storage and snapshot/fork benchmarks.

  • Adds a neon participant to benchmarks/storage/providers.ts with NEON_BUCKET, NEON_ENDPOINT, NEON_ACCESS_KEY_ID, and NEON_SECRET_ACCESS_KEY (plus optional NEON_REGION). It reuses the S3-compatible adapter for snapshot/fork, with optional NEON_SNAPSHOT_* overrides that fall back to the upload/download creds.
  • Adds bench:storage:neon and bench:snapshot-fork:neon scripts.
  • Adds neon to the storage and snapshot/fork benchmark matrices and vault-secret loader regexes.
  • Updates benchmarks/scripts/provider-vars.json with the Neon env var set.
  • Bumps @storagesdk/adapters to ^0.11.0 (the pending minor release that ships neon).

This PR is ready for review but depends on @storagesdk/adapters@0.11.0 being published; once it is, run pnpm install to update pnpm-lock.yaml before merging.

Link to Devin session: https://app.devin.ai/sessions/98223325d0664503bdea8a4f71ada5f5

Co-Authored-By: Noah Kiser <noah@computesdk.com>
@kisernl kisernl self-assigned this Aug 6, 2026
@devin-ai-integration

Copy link
Copy Markdown
Contributor

🤖 Devin AI Engineer

I'll be helping with this pull request! Here's what you should know:

✅ I will automatically:

  • Address comments on this PR. Add '(aside)' to your comment to have me ignore it.
  • Look at CI failures and help fix them

Note: I can only respond to comments from users who have write access to this repository.

⚙️ Control Options:

  • Disable automatic comment, CI, and merge conflict monitoring

Co-Authored-By: Noah Kiser <noah@computesdk.com>
@kisernl
kisernl marked this pull request as ready for review August 7, 2026 15:54
@open-cla

open-cla Bot commented Aug 7, 2026

Copy link
Copy Markdown

Contributor License Agreement

All contributors are covered by a CLA.

@superagent-security superagent-security Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Superagent found 2 security concern(s).

Comment thread .github/workflows/storage-benchmarks.yml Outdated
Comment thread .github/workflows/snapshot-fork-benchmarks.yml Outdated

@devin-ai-integration devin-ai-integration Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Devin Review found 2 potential issues.

Open in Devin Review

Comment thread benchmarks/scripts/provider-vars.json Outdated
Comment thread .github/workflows/storage-benchmarks.yml Outdated
…lows

Co-Authored-By: Noah Kiser <noah@computesdk.com>
@devin-ai-integration

Copy link
Copy Markdown
Contributor

Addressed the review comments in c555563:

  • Security scanner (.github/workflows/*): Replaced the broad NEON_[A-Z0-9_]* wildcard in both storage-benchmarks.yml and snapshot-fork-benchmarks.yml with explicit Neon env keys. snapshot-fork-benchmarks.yml includes the optional NEON_SNAPSHOT_* overrides; storage-benchmarks.yml includes the optional NEON_REGION override.
  • Devin Review (benchmarks/scripts/provider-vars.json): Regenerated the manifest with npx tsx benchmarks/scripts/gen-provider-vars.ts so the neon entries under storage and snapshot-fork match the requiredEnvVars in benchmarks/storage/providers.ts (four base credentials; optional overrides are intentionally excluded from the manifest).

Verification:

  • npx tsx benchmarks/scripts/gen-provider-vars.ts produces no diff against the committed provider-vars.json.
  • pnpm typecheck passes.
  • Superagent Security Scan is now green; only the Supply Chain Scan is still pending.

@kisernl
kisernl merged commit 33e8c61 into master Aug 7, 2026
6 checks passed
@github-actions

github-actions Bot commented Aug 7, 2026

Copy link
Copy Markdown
Contributor

Storage Benchmark Results

1MB Files

# Provider Score Download Throughput Upload Status
1 Azure Blob Storage 96.2 0.03s 307.9 Mbps 0.07s 10/10
2 Archil 95.7 0.04s 210.9 Mbps 0.20s 10/10
3 AWS S3 94.8 0.11s 78.3 Mbps 0.21s 10/10
4 Cloudflare R2 94.7 0.10s 82.6 Mbps 0.32s 10/10
5 Neon 94.6 0.12s 67.1 Mbps 0.24s 10/10
6 Vercel Blob 94.5 0.18s 47.3 Mbps 0.24s 10/10
7 Tensorlake 94.2 0.05s 152.9 Mbps 0.84s 10/10
8 Tigris 91.8 0.33s 26.9 Mbps 0.32s 10/10
9 Google Cloud Storage 0.0 0.00s 0.0 Mbps 0.00s 0/10

View full run · SVGs available as build artifacts

@github-actions

github-actions Bot commented Aug 7, 2026

Copy link
Copy Markdown
Contributor

Browser Benchmark Results

# Provider Score Create Connect Navigate Release Total Status
1 Kernel 97.9 0.03s 0.06s 0.09s 0.04s 0.22s 10/10
2 Tilion 97.1 0.09s 0.05s 0.10s 0.03s 0.28s 10/10
3 Browseruse 95.8 0.15s 0.11s 0.08s 0.04s 0.45s 10/10
4 Hyperbrowser 94.1 0.26s 0.14s 0.13s 0.09s 0.62s 10/10
5 Browserbase 93.5 0.17s 0.30s 0.14s 0.14s 0.77s 10/10
6 Notte 91.3 0.21s 0.50s 0.22s 0.09s 1.08s 10/10
7 Steel 74.6 0.48s 0.90s 0.14s 0.19s 2.73s 10/10

View full run · SVG available as build artifact

@github-actions

github-actions Bot commented Aug 7, 2026

Copy link
Copy Markdown
Contributor

Snapshot/Fork Benchmark Results

small dataset

# Provider Score Snapshot create Fork (snapshot) Fork (live) First read Status
1 Tensorlake 99.6 0.04s 0.36s 0.39s 0.12s 1/1
2 Tigris 99.3 0.30s 0.44s 0.65s 0.17s 1/1
3 Neon 99.0 0.53s 0.67s 1.17s 0.12s 1/1
4 Azure-blob 98.3 0.95s 1.16s 1.72s 0.04s 1/1
5 Cloudflare R2 78.1 9.70s 12.33s 32.78s 0.29s 1/1
6 Archil 0.0 0.00s 0.00s 0.00s 0.00s 0/1
7 AWS S3 0.0 0.00s 0.00s 0.00s 0.00s 0/1

View full run

@github-actions

github-actions Bot commented Aug 7, 2026

Copy link
Copy Markdown
Contributor

Browser Throughput Benchmark Results

# Provider Score APS (med) Task (med) Task (p95) Screenshot Status
1 Kernel 85.0 6.98/s 1.43s 2.32s 250ms 3/3
2 Browserbase 72.5 4.16/s 2.41s 3.02s 281ms 3/3
3 Browseruse 69.3 3.45/s 2.90s 2.94s 333ms 3/3
4 Tilion 67.8 3.17/s 3.16s 3.18s 254ms 3/3
5 Hyperbrowser 62.5 2.43/s 4.11s 5.24s 500ms 3/3
6 Steel 53.9 1.37/s 7.30s 7.75s 714ms 3/3
7 Notte 52.4 1.34/s 7.47s 9.48s 902ms 3/3

View full run · SVG available as build artifact

@github-actions

github-actions Bot commented Aug 7, 2026

Copy link
Copy Markdown
Contributor

Sandbox Benchmark Results

Sequential

# Provider Score Median TTI P95 P99 Status
1 isorun 99.7 0.02s 0.04s 0.04s 10/10
2 createos 99.0 0.08s 0.12s 0.12s 10/10
3 declaw 99.0 0.02s 0.22s 0.22s 10/10
4 arker 99.0 0.08s 0.13s 0.13s 10/10
5 northflank 98.4 0.10s 0.25s 0.25s 10/10
6 archil 97.8 0.17s 0.29s 0.29s 10/10
7 lightning 97.4 0.17s 0.39s 0.39s 10/10
8 vercel 96.0 0.33s 0.49s 0.49s 10/10
9 blaxel 95.7 0.23s 0.74s 0.74s 10/10
10 beam 95.5 0.09s 0.98s 0.98s 10/10
11 superserve 95.2 0.37s 0.65s 0.65s 10/10
12 upstash 94.6 0.46s 0.67s 0.67s 10/10
13 modal 93.8 0.57s 0.70s 0.70s 10/10
14 cloud-run 93.5 0.57s 0.76s 0.76s 10/10
15 runloop 93.4 0.45s 0.97s 0.97s 10/10
16 tensorlake 92.7 0.72s 0.75s 0.75s 10/10
17 tenki 91.2 0.70s 1.14s 1.14s 10/10
18 sandbox0 90.9 0.78s 1.11s 1.11s 10/10
19 e2b 88.7 0.88s 1.51s 1.51s 10/10
20 sail 88.3 0.67s 1.93s 1.93s 10/10
21 opencomputer 77.6 0.47s 4.90s 4.90s 10/10
22 cloudflare 72.6 2.27s 3.44s 3.44s 10/10
23 daytona 49.5 0.83s 59.28s 59.28s 9/10
24 codesandbox 45.4 2.43s 23.28s 23.28s 10/10
25 hopx 0.0 0.00s 0.00s 0.00s 0/10
26 mosaic 0.0 0.00s 0.00s 0.00s 0/10

Staggered

# Provider Score Median TTI P95 P99 Status
1 declaw 99.6 0.02s 0.07s 0.07s 10/10
2 createos 99.0 0.08s 0.12s 0.12s 10/10
3 arker 98.9 0.08s 0.15s 0.15s 10/10
4 northflank 98.8 0.09s 0.15s 0.15s 10/10
5 archil 98.3 0.15s 0.21s 0.21s 10/10
6 beam 98.1 0.10s 0.32s 0.32s 10/10
7 lightning 97.9 0.15s 0.30s 0.30s 10/10
8 opencomputer 97.9 0.09s 0.40s 0.40s 10/10
9 blaxel 97.1 0.26s 0.33s 0.33s 10/10
10 superserve 96.7 0.29s 0.38s 0.38s 10/10
11 upstash 95.0 0.47s 0.54s 0.54s 10/10
12 vercel 94.8 0.39s 0.72s 0.72s 10/10
13 cloud-run 94.0 0.57s 0.64s 0.64s 10/10
14 modal 93.8 0.57s 0.71s 0.71s 10/10
15 runloop 93.3 0.59s 0.80s 0.80s 10/10
16 tenki 91.8 0.68s 1.03s 1.03s 10/10
17 tensorlake 91.4 0.70s 1.10s 1.10s 10/10
18 sail 89.5 0.76s 1.47s 1.47s 10/10
19 e2b 89.4 0.93s 1.26s 1.26s 10/10
20 sandbox0 88.0 1.03s 1.46s 1.46s 10/10
21 isorun 83.9 0.02s 4.00s 4.00s 10/10
22 cloudflare 77.0 2.09s 2.62s 2.62s 10/10
23 codesandbox 75.3 2.34s 2.68s 2.68s 10/10
24 daytona 54.4 0.93s 21.44s 21.44s 10/10
25 hopx 0.0 0.00s 0.00s 0.00s 0/10
26 mosaic 0.0 0.00s 0.00s 0.00s 0/10

Burst

# Provider Score Median TTI P95 P99 Status
1 declaw 99.6 0.04s 0.04s 0.04s 10/10
2 createos 98.7 0.13s 0.14s 0.14s 10/10
3 beam 98.6 0.12s 0.15s 0.15s 10/10
4 arker 98.4 0.14s 0.20s 0.20s 10/10
5 lightning 98.0 0.19s 0.22s 0.22s 10/10
6 northflank 97.6 0.21s 0.27s 0.27s 10/10
7 blaxel 97.3 0.24s 0.30s 0.30s 10/10
8 archil 97.3 0.19s 0.40s 0.40s 10/10
9 opencomputer 97.0 0.23s 0.40s 0.40s 10/10
10 superserve 96.5 0.34s 0.36s 0.36s 10/10
11 vercel 95.3 0.41s 0.56s 0.56s 10/10
12 upstash 93.9 0.60s 0.61s 0.61s 10/10
13 cloud-run 93.9 0.56s 0.70s 0.70s 10/10
14 runloop 93.1 0.63s 0.77s 0.77s 10/10
15 modal 92.7 0.64s 0.86s 0.86s 10/10
16 tensorlake 92.3 0.77s 0.78s 0.78s 10/10
17 tenki 89.0 0.95s 1.32s 1.32s 10/10
18 e2b 88.1 1.03s 1.44s 1.44s 10/10
19 sail 85.3 1.47s 1.47s 1.47s 10/10
20 sandbox0 81.4 1.79s 1.95s 1.95s 10/10
21 codesandbox 70.2 2.73s 3.34s 3.34s 10/10
22 cloudflare 68.2 2.64s 3.99s 3.99s 10/10
23 isorun 59.8 0.04s 10.58s 10.58s 10/10
24 daytona 54.4 0.94s 10.66s 10.66s 10/10
25 hopx 0.0 0.00s 0.00s 0.00s 0/10
26 mosaic 0.0 0.00s 0.00s 0.00s 0/10

View full run · SVGs available as build artifacts

@github-actions

github-actions Bot commented Aug 7, 2026

Copy link
Copy Markdown
Contributor

Sandbox Dax Benchmark Results

Provider Phases Total Prepare Clone Install Typecheck Status
archil 0/7 0.00s 0.00s 0.00s 0.00s 0.00s 0/1 OK
arker 7/7 59.73s 4.15s 2.54s 11.46s 35.91s 1/1 OK
beam 0/7 0.00s 0.00s 0.00s 0.00s 0.00s 0/1 OK
blaxel 6/7 42.89s 1.84s 2.23s 11.08s 0.00s 0/1 OK
cloud-run 2/7 1.30s 0.46s 0.00s 0.00s 0.00s 0/1 OK
cloudflare 0/7 0.00s 0.00s 0.00s 0.00s 0.00s 0/1 OK
codesandbox 0/7 0.00s 0.00s 0.00s 0.00s 0.00s 0/1 OK
createos 7/7 51.05s 2.87s 2.15s 10.76s 26.72s 1/1 OK
daytona 7/7 69.74s 2.57s 3.74s 13.97s 35.87s 1/1 OK
declaw 7/7 72.30s 5.99s 2.08s 12.29s 43.79s 1/1 OK
e2b 6/7 31.57s 8.13s 4.53s 11.39s 0.00s 0/1 OK
hopx -- 0.00s 0.00s 0.00s 0.00s 0.00s 0/1 OK
isorun 7/7 96.50s 0.97s 1.67s 10.47s 28.55s 1/1 OK
lightning 7/7 37.23s 3.63s 2.69s 11.79s 14.06s 1/1 OK
modal 6/7 25.75s 4.14s 2.66s 12.94s 0.00s 0/1 OK
mosaic -- 0.00s 0.00s 0.00s 0.00s 0.00s 0/1 OK
namespace 7/7 46.52s 5.00s 4.96s 8.09s 22.52s 1/1 OK
northflank -- 0.00s 0.00s 0.00s 0.00s 0.00s 0/1 OK
opencomputer 6/7 12.92s 4.04s 1.94s 5.42s 0.00s 0/1 OK
runloop 2/7 116.22s 0.00s 0.00s 26.55s 50.73s 1/1 OK
sail 7/7 310.93s 2.15s 3.14s 28.69s 269.27s 1/1 OK
sandbox0 7/7 135.87s 4.30s 7.53s 74.03s 34.34s 1/1 OK
superserve 7/7 101.38s 6.38s 4.66s 23.63s 62.84s 1/1 OK
tenki 7/7 67.97s 3.48s 4.21s 18.07s 36.13s 1/1 OK
tensorlake 7/7 54.99s 10.49s 1.69s 10.59s 27.93s 1/1 OK
upstash 7/7 50.82s 4.04s 2.29s 9.37s 24.69s 1/1 OK
vercel 7/7 70.41s 16.48s 2.11s 11.65s 33.88s 1/1 OK

View full run

@github-actions

github-actions Bot commented Aug 7, 2026

Copy link
Copy Markdown
Contributor

AI Gateway Benchmark Results

# Gateway Score Cold E2E Warm TTFT Tok/sec Status
1 Anthropic (direct) 91.2 531ms 503ms 89.8 20/20
2 LLM Gateway 91.1 638ms 567ms 89.6 20/20
3 Cloudflare AI Gateway 90.9 560ms 571ms 85.5 20/20
4 Concentrate AI 90.4 859ms 666ms 89.8 20/20
5 OpenRouter 90.2 710ms 623ms 88.7 20/20
6 Pydantic AI Gateway 89.9 578ms 723ms 87.4 20/20
7 Vercel AI Gateway 82.5 711ms 584ms 93.1 20/20

View full run · SVG available as a build artifact

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant