fix(tools): read the registry's hosted-key verdict instead of re-deriving it

The metering gate re-derived "did Sim's key pay" from the tool's hosting config
and whether the caller omitted the key. That cannot see a workspace or
organization BYOK key, which `injectHostedKeyIfNeeded` injects while reporting
`isUsingHostedKey: false` — the org's own key, not billable — so the derivation
called a BYOK call hosted.

The registry's verdict does propagate, by one path. On a tool with `hosting`,
`output.cost` has a single writer, `applyHostedKeyCostToResult`, and it runs only
under `isUsingHostedKey && success`. A BYOK call, a caller-keyed call, and a
failed call all leave the field absent. The gate now reads that: `hosting`
present, success, cost present. The derivation stays for the one question it
can answer before dispatch — a parameter Sim will fill is not missing.

The reading rests on hosted tools not reporting their own cost in that field,
which is true of all 131 today and now pinned by the reachability audit, so a
future hosted tool that self-reports fails CI rather than billing BYOK calls.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

Author: icecrasher321

apps/sim/lib/tool-execution/application/execute-tool.test.ts

@@ -606,17 +606,40 @@ describe('executeToolForCaller', () => {
     expect(mocks.recordUsage).not.toHaveBeenCalled()
   })
 
+  /**
+   * Mirrors the real registry contract: a caller's own key means
+   * `isUsingHostedKey` is false, so `applyHostedKeyCostToResult` never runs and
+   * no `output.cost` is written. The meter reads that absence as the verdict.
+   */
   it('does not bill when the caller brought their own key', async () => {
     mocks.executeRegistryTool.mockResolvedValue({
       success: true,
-      output: { cost: { total: 0.004 } },
+      output: { markdown: '# Hi' },
     })
 
     await run({ input: { url: 'https://a.co', apiKey: 'sk-mine' } })
 
     expect(mocks.recordUsage).not.toHaveBeenCalled()
   })
 
+  /**
+   * The BYOK shape. The registry injected the org's own key and returned
+   * `isUsingHostedKey: false`, so it wrote no `output.cost` — and the caller
+   * omitted the key, which a pre-dispatch derivation reads as "Sim's". Only the
+   * registry's verdict, carried by the presence of the cost it alone writes,
+   * gets this right.
+   */
+  it('does not bill a BYOK call, where the key was omitted but Sim did not pay', async () => {
+    mocks.executeRegistryTool.mockResolvedValue({
+      success: true,
+      output: { markdown: '# Hi' },
+    })
+
+    await run({ input: { url: 'https://a.co' } })
+
+    expect(mocks.recordUsage).not.toHaveBeenCalled()
+  })
+
   it('does not bill a failed call', async () => {
     mocks.executeRegistryTool.mockResolvedValue({
       success: false,

apps/sim/lib/tool-execution/application/execute-tool.ts

@@ -49,9 +49,10 @@ export interface ExecuteToolResult {
  * deployment hosts keys, any `enabled` predicate accepts these params, and the
  * caller has not brought a key of their own — which wins where present.
  *
- * Read twice, for the two questions that both turn on it. A required parameter
- * Sim fills is not missing, and spend is only Sim's to bill when Sim's key paid
- * for it.
+ * Pre-dispatch only, for the required-input exemption: a parameter Sim will
+ * fill is not missing. It is deliberately NOT the metering gate — it cannot see
+ * a BYOK key, which the registry injects while reporting the call as *not*
+ * hosted, so after dispatch the registry's own verdict is read instead.
  */
 function hostedKeyParamFor(
   tool: ExecutableToolConfig,
@@ -342,14 +343,26 @@ export const executeToolForCaller = defineAuthorizedWorkspaceUseCase({
     })
 
     /**
-     * Only a successful call that spent Sim's key. `output.cost` is not a
-     * hosted-key marker: `knowledge_upload_chunk` and the enrichment runner
-     * report their own cost there, and billing those as `api-tool` spend would
-     * charge a second time for something already metered elsewhere. The registry
-     * writes hosted-key cost under the same two conditions
-     * (`hostedKeyInfo.isUsingHostedKey && finalResult.success`).
+     * Only a successful call that spent Sim's key — and that verdict is the
+     * registry's, not re-derived here.
+     *
+     * The registry decides whether Sim's key was used inside
+     * `injectHostedKeyIfNeeded`, and a workspace or organization BYOK key is
+     * one of the ways it decides *no*: the org's own key is injected and
+     * `isUsingHostedKey` is false. A pre-dispatch derivation cannot see that
+     * (it would need the BYOK lookup), so an earlier version of this gate
+     * treated every omitted key as Sim's and was wrong for BYOK.
+     *
+     * The verdict does propagate, by one path: on a tool with `hosting`,
+     * `output.cost` has a single writer, `applyHostedKeyCostToResult`, and it
+     * runs only under `hostedKeyInfo.isUsingHostedKey && finalResult.success`.
+     * So `hosting` present + success + cost present *is* "Sim's key paid".
+     * `hosting` is checked because tools without it — `knowledge_upload_chunk`,
+     * the enrichment runner — report their own cost in that field and are
+     * metered elsewhere. That no hosted tool does the same is what
+     * `check-tool-param-reachability` now pins.
      */
-    if (result.success && hostedKeyParamFor(tool, params)) {
+    if (result.success && tool.hosting) {
       await meterHostedKeySpend({
         callId,
         toolId,

scripts/check-tool-param-reachability.ts

@@ -85,6 +85,27 @@ interface Finding {
   reason: string
 }
 
+/**
+ * A hosted tool must not declare its own `cost` output.
+ *
+ * Direct execution (`POST /api/v2/tools/{toolId}/execute`) bills hosted-key
+ * spend by reading `output.cost` on a tool with `hosting` — because on such a
+ * tool that field has exactly one writer, `applyHostedKeyCostToResult`, which
+ * runs only when the registry actually used Sim's key on a successful call. A
+ * BYOK call leaves it absent and so is not billed. A hosted tool that also
+ * reported its own cost there would break that reading: its self-reported
+ * number would bill as Sim's spend on a BYOK call or a caller-keyed call. Tools
+ * without `hosting` may report cost freely; the meter never looks at them.
+ */
+function findHostedToolsReportingCost(): string[] {
+  return Object.entries(tools as Record<string, ToolConfig>)
+    .filter(
+      ([, config]) => config.hosting && config.outputs && Object.hasOwn(config.outputs, 'cost')
+    )
+    .map(([toolId]) => toolId)
+    .sort()
+}
+
 function findUnreachableParams(): Finding[] {
   const findings: Finding[] = []
 
@@ -129,6 +150,17 @@ function findUnreachableParams(): Finding[] {
 function main(): void {
   const findings = findUnreachableParams()
   const toolCount = Object.keys(tools).length
+  const costReporters = findHostedToolsReportingCost()
+
+  if (costReporters.length > 0) {
+    console.error('Tool parameter reachability audit failed:\n')
+    for (const toolId of costReporters) {
+      console.error(
+        `  ${toolId} — declares hosting AND a 'cost' output; direct execution reads output.cost on a hosted tool as "Sim's key paid", so a self-reported cost would bill BYOK and caller-keyed calls`
+      )
+    }
+    process.exit(1)
+  }
 
   if (findings.length === 0) {
     console.log(