Skip to content

Commit e20ad10

Browse files
SK-3133 add unary-vs-bulk guidance and concurrency sizing formula to flowvault README
- Batching and concurrency: a starting-point sizing formula (N_cpu x U_cpu x (1 + W/C)) for ..._CONCURRENCY_LIMIT, with guidance to benchmark and raise incrementally rather than jump to the formula's theoretical max. - Unary vs. bulk: when each fits, and the N < batchSize trap where bulk resolves to concurrency=1 with none of the batching benefit. Both folded into the existing headings, no new sections added. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
1 parent e70e792 commit e20ad10

1 file changed

Lines changed: 17 additions & 0 deletions

File tree

flowvault/README.md

Lines changed: 17 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -613,6 +613,19 @@ INSERT_CONCURRENCY_LIMIT=5
613613

614614
The 100,000-item ceiling per bulk call is a separate, fixed limit and is not configurable.
615615

616+
`..._CONCURRENCY_LIMIT` has no single right value — it depends on the CPU available to your process and how much of each call's time is spent waiting on the network versus doing work on the client:
617+
618+
```
619+
concurrency ≈ N_cpu × U_cpu × (1 + W/C)
620+
```
621+
622+
- `N_cpu` — CPU cores available to the process
623+
- `U_cpu` — target CPU utilization, between 0 and 1
624+
- `W` — wait time per call, roughly your observed API latency
625+
- `C` — compute time per call on the client side — small, on the order of a few milliseconds for this SDK
626+
627+
As a starting point, a single CPU core is generally sufficient up to about 20 concurrent virtual users (VUs); beyond that, consider increasing CPU capacity and tuning concurrency accordingly — dual- or quad-core configurations are a good starting point for higher-throughput workloads. Treat the formula as a baseline, not a final answer: benchmark with your actual API latency and workload before raising concurrency further. A high `W/C` ratio can suggest a larger number than a shared client-side resource — like the connection pool — can actually sustain once its own overhead grows under load, so raise `..._CONCURRENCY_LIMIT` incrementally and confirm latency stays flat at each step rather than jumping straight to a theoretical maximum.
628+
616629
# VaultController — Unary operations
617630

618631
Alongside the bulk methods, `VaultController` exposes five **unary** operations. Each sends exactly one API call and hands the result straight back:
@@ -643,6 +656,10 @@ Everything the bulk machinery adds — batching, concurrency, the payload ceilin
643656
| Retry helper | `getRecordsToRetry()` / `getTokensToRetry()` | None — filter the records yourself, see [Retrying the failed records](#retrying-the-failed-records) |
644657
| Per-item `getHttpCode()` / `getError()` | Yes | Yes, on every unary operation |
645658

659+
Both accept as many records as you pass — the difference above is entirely about how the SDK makes HTTP calls and manages concurrency, not how much data a single call can carry. Unary is best suited for single-event or low-volume ingestion, interactive or user-facing requests where an immediate result is required, and applications that already have their own concurrency or job-management mechanism (e.g. a fixed worker pool) — layering the bulk machinery on top would just duplicate it. Bulk is best suited for large datasets — imports, backfills, ETL, and data migration workloads — and for streaming ingestion once buffered into a window (flush every `batchSize`-or-more records, or on a timer), so bulk can parallelize the dispatch of each flushed chunk.
660+
661+
A bulk call sent with fewer records than `batchSize` (default 50) still produces exactly one batch — `concurrency` resolves to 1 regardless of `..._CONCURRENCY_LIMIT` — so there's no batching benefit, only the overhead of the bulk machinery on top. Use unary instead for calls at that size.
662+
646663
## Vault type support
647664

648665
The same distinction as [Schema vs. schemaless vaults](#schema-vs-schemaless-vaults) applies. Four of the five unary operations address records inside a table, so they only make sense against a structured vault:

0 commit comments

Comments
 (0)