You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
SK-3133 add unary-vs-bulk guidance and concurrency sizing formula to flowvault README
- Batching and concurrency: a starting-point sizing formula
(N_cpu x U_cpu x (1 + W/C)) for ..._CONCURRENCY_LIMIT, with guidance
to benchmark and raise incrementally rather than jump to the
formula's theoretical max.
- Unary vs. bulk: when each fits, and the N < batchSize trap where
bulk resolves to concurrency=1 with none of the batching benefit.
Both folded into the existing headings, no new sections added.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Copy file name to clipboardExpand all lines: flowvault/README.md
+17Lines changed: 17 additions & 0 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -613,6 +613,19 @@ INSERT_CONCURRENCY_LIMIT=5
613
613
614
614
The 100,000-item ceiling per bulk call is a separate, fixed limit and is not configurable.
615
615
616
+
`..._CONCURRENCY_LIMIT` has no single right value — it depends on the CPU available to your process and how much of each call's time is spent waiting on the network versus doing work on the client:
617
+
618
+
```
619
+
concurrency ≈ N_cpu × U_cpu × (1 + W/C)
620
+
```
621
+
622
+
-`N_cpu` — CPU cores available to the process
623
+
-`U_cpu` — target CPU utilization, between 0 and 1
624
+
-`W` — wait time per call, roughly your observed API latency
625
+
-`C` — compute time per call on the client side — small, on the order of a few milliseconds for this SDK
626
+
627
+
As a starting point, a single CPU core is generally sufficient up to about 20 concurrent virtual users (VUs); beyond that, consider increasing CPU capacity and tuning concurrency accordingly — dual- or quad-core configurations are a good starting point for higher-throughput workloads. Treat the formula as a baseline, not a final answer: benchmark with your actual API latency and workload before raising concurrency further. A high `W/C` ratio can suggest a larger number than a shared client-side resource — like the connection pool — can actually sustain once its own overhead grows under load, so raise `..._CONCURRENCY_LIMIT` incrementally and confirm latency stays flat at each step rather than jumping straight to a theoretical maximum.
628
+
616
629
# VaultController — Unary operations
617
630
618
631
Alongside the bulk methods, `VaultController` exposes five **unary** operations. Each sends exactly one API call and hands the result straight back:
@@ -643,6 +656,10 @@ Everything the bulk machinery adds — batching, concurrency, the payload ceilin
643
656
| Retry helper |`getRecordsToRetry()` / `getTokensToRetry()`| None — filter the records yourself, see [Retrying the failed records](#retrying-the-failed-records)|
644
657
| Per-item `getHttpCode()` / `getError()`| Yes | Yes, on every unary operation |
645
658
659
+
Both accept as many records as you pass — the difference above is entirely about how the SDK makes HTTP calls and manages concurrency, not how much data a single call can carry. Unary is best suited for single-event or low-volume ingestion, interactive or user-facing requests where an immediate result is required, and applications that already have their own concurrency or job-management mechanism (e.g. a fixed worker pool) — layering the bulk machinery on top would just duplicate it. Bulk is best suited for large datasets — imports, backfills, ETL, and data migration workloads — and for streaming ingestion once buffered into a window (flush every `batchSize`-or-more records, or on a timer), so bulk can parallelize the dispatch of each flushed chunk.
660
+
661
+
A bulk call sent with fewer records than `batchSize` (default 50) still produces exactly one batch — `concurrency` resolves to 1 regardless of `..._CONCURRENCY_LIMIT` — so there's no batching benefit, only the overhead of the bulk machinery on top. Use unary instead for calls at that size.
662
+
646
663
## Vault type support
647
664
648
665
The same distinction as [Schema vs. schemaless vaults](#schema-vs-schemaless-vaults) applies. Four of the five unary operations address records inside a table, so they only make sense against a structured vault:
0 commit comments