Skip to content

Commit ded98d3

Browse files
committed
test(tools): single discovery sweep, and record why the assertions are exact
cubic found toolsWithoutPathParams re-ran the whole barrel sweep although every suite already calls discoverPathParams once. Discovery builds a URL for every tool, branch assignment and declared parameter, so that doubled the most expensive part of each suite for a list already in hand. The inventory now comes back from the same sweep as withoutPathParams and the standalone helper is gone. Also records, in both the shared harness header and the Supabase suite, why the assertions pin exact encoded output and exact error text. Those guards live in url-path.ts, owned by the PR this branch is rebased onto, so their behaviour changes land underneath this suite without touching a line of it. That has happened twice — segment trimming dropped, then the empty-segment check narrowing from !segment.trim() to !segment — and only the exact assertions caught either. A suite asserting just toThrow() would have gone green through both, and the second is a silent correctness change in either direction.
1 parent bc5b65c commit ded98d3

7 files changed

Lines changed: 72 additions & 54 deletions

File tree

apps/sim/tools/__tests__/path-safety.ts

Lines changed: 52 additions & 40 deletions
Original file line numberDiff line numberDiff line change
@@ -34,6 +34,29 @@
3434
* normalization `fetch` performs — instead of string-matching the template
3535
* output. String matching is exactly what let dot-segment traversal through:
3636
* the template looks correct and the parser rewrites it afterwards.
37+
*
38+
* **Do not loosen these assertions into `toThrow()` or `toContain()`.** They
39+
* deliberately pin the *exact* encoded output and the *exact* error text, and
40+
* that precision is load-bearing rather than fussy: these guards live in
41+
* `tools/url-path.ts`, which belongs to a different PR that this branch is
42+
* rebased onto, so changes to them land *underneath* this suite without
43+
* touching a line of it.
44+
*
45+
* That has happened twice, and both times only the exact assertions noticed:
46+
*
47+
* - `safeUrlPath` stopped trimming each segment, so a storage key's interior
48+
* whitespace began surviving to the wire. Caught by an equality assertion on
49+
* the encoded output.
50+
* - Its empty-segment check narrowed from `!segment.trim()` to `!segment`, so
51+
* `a/ /b` became legal while `a//b` stayed rejected. Caught by an assertion
52+
* on the exact error text.
53+
*
54+
* A suite asserting only "it throws" would have gone green through both, and
55+
* the second one is a silent correctness change in either direction — permitting
56+
* `a//b` retargets the request at a different object, while rejecting `a/ /b`
57+
* makes a real object key permanently unreachable. The precision is what turns
58+
* an upstream edit into a failing test instead of a behaviour change nobody
59+
* sees.
3760
*/
3861
import { getErrorMessage } from '@sim/utils/errors'
3962
import { expect, it } from 'vitest'
@@ -300,6 +323,32 @@ export function discoverPathParams(
300323
covered: PathParam[]
301324
unbuildable: UnbuildableTool[]
302325
undiscoverable: UndiscoverableParam[]
326+
/**
327+
* Every tool of the service that contributes no (tool, parameter) pair.
328+
*
329+
* Returned from the same sweep rather than recomputed, because discovery
330+
* builds a URL for every tool, every branch assignment and every declared
331+
* parameter — running it twice per suite doubled that for a list already in
332+
* hand.
333+
*
334+
* The enumeration is deliberately **looser** than `covered`. That list only
335+
* holds tools whose `request.url` is a function, since discovery has to call
336+
* it. Filtering the inventory the same way made three categories invisible to
337+
* *both* sides and let the pin pass vacuously — `box_create_folder` declares
338+
* `url` as a plain string and `box_upload_file` is an `InternalToolConfig`
339+
* with no `request`, so neither appeared in the covered pairs *or* in the
340+
* pinned set. Eleven tools across four services were invisible that way. So
341+
* this walks every export carrying the service id prefix, whatever shape its
342+
* request takes, and each suite pins the result exactly: a tool that gains a
343+
* guarded path parameter leaves the list, the assertion fails, and someone
344+
* looks.
345+
*
346+
* An `InternalToolConfig` stays here permanently — its URL is built in
347+
* `lib/internal/**`, which this suite cannot drive. Pinning it proves it is
348+
* accounted for, not that it is traversal-safe; that coverage comes from
349+
* direct unit tests on the helper it shares.
350+
*/
351+
withoutPathParams: string[]
303352
} {
304353
const covered: PathParam[] = []
305354
const unbuildable: UnbuildableTool[] = []
@@ -376,51 +425,14 @@ export function discoverPathParams(
376425
}
377426
}
378427

379-
return { covered, unbuildable, undiscoverable }
380-
}
381-
382-
/**
383-
* Lists every tool of a service that contributes **no** (tool, parameter) pair.
384-
*
385-
* Each suite pins this set exactly, so a tool cannot leave path coverage
386-
* unnoticed: if one ever gains a guarded path parameter it becomes a covered
387-
* pair, the set shrinks, and the assertion fails until someone looks.
388-
*
389-
* The enumeration is deliberately **looser** than {@link discoverPathParams}.
390-
* That function can only drive a tool whose `request.url` is a function, since
391-
* it has to call it. Filtering the inventory the same way would make three
392-
* whole categories invisible to *both* sides and let the pin pass vacuously —
393-
* which is exactly what happened before: `box_create_folder` declares
394-
* `url` as a plain **string**, and `box_upload_file` is an `InternalToolConfig`
395-
* with no `request` at all, so neither appeared in the covered pairs *or* in
396-
* the pinned set, and `toEqual(['box_search'])` passed precisely because they
397-
* could not be seen. Eleven tools across four services were invisible that way.
398-
*
399-
* So this walks every export whose `id` carries the service prefix, whatever
400-
* shape its request takes, and reports the ones no pair covers. The pinned list
401-
* then states the real inventory, and each entry has to be justified as one of:
402-
*
403-
* - a genuinely static or query-string-only URL (`box_search`);
404-
* - a `url` declared as a constant string (`box_create_folder`);
405-
* - an `InternalToolConfig` whose URL is built in `lib/internal/**`
406-
* (`supabase_storage_upload`). **These are outside what this suite can
407-
* reach**, and are covered instead by direct unit tests on the helper they
408-
* use — see the `encodeStoragePath` / `encodeStorageSegment` describes in
409-
* `supabase/path_safety.test.ts`.
410-
*/
411-
export function toolsWithoutPathParams(
412-
barrel: Record<string, unknown>,
413-
idPrefix: string,
414-
fixed: Record<string, unknown> = {}
415-
): string[] {
416-
const { covered } = discoverPathParams(barrel, idPrefix, fixed)
417428
const withParams = new Set(covered.map(({ tool }) => tool.id))
418-
419-
return Object.values(barrel)
429+
const withoutPathParams = Object.values(barrel)
420430
.map((value) => (value as { id?: unknown } | null)?.id)
421431
.filter((id): id is string => typeof id === 'string' && id.startsWith(idPrefix))
422432
.filter((id) => !withParams.has(id))
423433
.sort()
434+
435+
return { covered, unbuildable, undiscoverable, withoutPathParams }
424436
}
425437

426438
/**

apps/sim/tools/box/path_safety.test.ts

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -13,7 +13,6 @@ import {
1313
discoverPathParams,
1414
itPassesLegitimateValues,
1515
itResistsTraversal,
16-
toolsWithoutPathParams,
1716
} from '@/tools/__tests__/path-safety'
1817
import * as boxTools from '@/tools/box/index'
1918

@@ -37,6 +36,7 @@ const {
3736
covered: PATH_PARAMS,
3837
unbuildable: UNBUILDABLE,
3938
undiscoverable: UNDISCOVERABLE,
39+
withoutPathParams: WITHOUT_PATH_PARAMS,
4040
} = discoverPathParams(boxTools, 'box_')
4141

4242
describe('box path-id traversal safety', () => {
@@ -49,7 +49,7 @@ describe('box path-id traversal safety', () => {
4949
})
5050

5151
it('leaves only genuinely static-URL tools without a path parameter', () => {
52-
expect(toolsWithoutPathParams(boxTools, 'box_')).toEqual(STATIC_URL_TOOLS)
52+
expect(WITHOUT_PATH_PARAMS).toEqual(STATIC_URL_TOOLS)
5353
})
5454

5555
it('covers every parameter that reaches a URL path segment', () => {

apps/sim/tools/box_sign/path_safety.test.ts

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -22,7 +22,6 @@ import {
2222
discoverPathParams,
2323
itPassesLegitimateValues,
2424
itResistsTraversal,
25-
toolsWithoutPathParams,
2625
} from '@/tools/__tests__/path-safety'
2726
import * as boxSignTools from '@/tools/box_sign/index'
2827

@@ -49,6 +48,7 @@ const {
4948
covered: PATH_PARAMS,
5049
unbuildable: UNBUILDABLE,
5150
undiscoverable: UNDISCOVERABLE,
51+
withoutPathParams: WITHOUT_PATH_PARAMS,
5252
} = discoverPathParams(boxSignTools, 'box_sign_')
5353

5454
describe('box sign path-id traversal safety', () => {
@@ -61,7 +61,7 @@ describe('box sign path-id traversal safety', () => {
6161
})
6262

6363
it('leaves only genuinely static-URL tools without a path parameter', () => {
64-
expect(toolsWithoutPathParams(boxSignTools, 'box_sign_')).toEqual(STATIC_URL_TOOLS)
64+
expect(WITHOUT_PATH_PARAMS).toEqual(STATIC_URL_TOOLS)
6565
})
6666

6767
it('covers every parameter that reaches a URL path segment', () => {

apps/sim/tools/google_bigquery/path_safety.test.ts

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -15,7 +15,6 @@ import {
1515
discoverPathParams,
1616
itPassesLegitimateValues,
1717
itResistsTraversal,
18-
toolsWithoutPathParams,
1918
} from '@/tools/__tests__/path-safety'
2019
import * as bigQueryTools from '@/tools/google_bigquery/index'
2120
import { canonicalBigQueryId, strictBigQueryPathSegment } from '@/tools/google_bigquery/utils'
@@ -88,6 +87,7 @@ const {
8887
covered: PATH_PARAMS,
8988
unbuildable: UNBUILDABLE,
9089
undiscoverable: UNDISCOVERABLE,
90+
withoutPathParams: WITHOUT_PATH_PARAMS,
9191
} = discoverPathParams(bigQueryTools, 'google_bigquery_')
9292

9393
describe('bigquery path-id traversal safety', () => {
@@ -100,7 +100,7 @@ describe('bigquery path-id traversal safety', () => {
100100
})
101101

102102
it('leaves only genuinely static-URL tools without a path parameter', () => {
103-
expect(toolsWithoutPathParams(bigQueryTools, 'google_bigquery_')).toEqual(STATIC_URL_TOOLS)
103+
expect(WITHOUT_PATH_PARAMS).toEqual(STATIC_URL_TOOLS)
104104
})
105105

106106
it('covers every parameter that reaches a URL path segment', () => {

apps/sim/tools/google_contacts/path_safety.test.ts

Lines changed: 2 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -15,7 +15,6 @@ import {
1515
discoverPathParams,
1616
itPassesLegitimateValues,
1717
itResistsTraversal,
18-
toolsWithoutPathParams,
1918
} from '@/tools/__tests__/path-safety'
2019
import * as googleContactsTools from '@/tools/google_contacts/index'
2120

@@ -48,6 +47,7 @@ const {
4847
covered: PATH_PARAMS,
4948
unbuildable: UNBUILDABLE,
5049
undiscoverable: UNDISCOVERABLE,
50+
withoutPathParams: WITHOUT_PATH_PARAMS,
5151
} = discoverPathParams(googleContactsTools, 'google_contacts_')
5252

5353
describe('google contacts resourceName traversal safety', () => {
@@ -60,9 +60,7 @@ describe('google contacts resourceName traversal safety', () => {
6060
})
6161

6262
it('leaves only genuinely static-URL tools without a path parameter', () => {
63-
expect(toolsWithoutPathParams(googleContactsTools, 'google_contacts_')).toEqual(
64-
STATIC_URL_TOOLS
65-
)
63+
expect(WITHOUT_PATH_PARAMS).toEqual(STATIC_URL_TOOLS)
6664
})
6765

6866
it('covers every parameter that reaches a URL path segment', () => {

apps/sim/tools/google_drive/path_safety.test.ts

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -17,7 +17,6 @@ import {
1717
discoverPathParams,
1818
itPassesLegitimateValues,
1919
itResistsTraversal,
20-
toolsWithoutPathParams,
2120
} from '@/tools/__tests__/path-safety'
2221
import * as googleDriveTools from '@/tools/google_drive/index'
2322

@@ -56,6 +55,7 @@ const {
5655
covered: PATH_PARAMS,
5756
unbuildable: UNBUILDABLE,
5857
undiscoverable: UNDISCOVERABLE,
58+
withoutPathParams: WITHOUT_PATH_PARAMS,
5959
} = discoverPathParams(googleDriveTools, 'google_drive_')
6060

6161
describe('google drive path-id traversal safety', () => {
@@ -68,7 +68,7 @@ describe('google drive path-id traversal safety', () => {
6868
})
6969

7070
it('leaves only genuinely static-URL tools without a path parameter', () => {
71-
expect(toolsWithoutPathParams(googleDriveTools, 'google_drive_')).toEqual(STATIC_URL_TOOLS)
71+
expect(WITHOUT_PATH_PARAMS).toEqual(STATIC_URL_TOOLS)
7272
})
7373

7474
it('covers every parameter that reaches a URL path segment', () => {

apps/sim/tools/supabase/path_safety.test.ts

Lines changed: 10 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -15,13 +15,20 @@
1515
* A storage key legitimately contains `/`, so the fix could not be
1616
* `safeUrlPathSegment`: it is `safeUrlPath`, which keeps the separator
1717
* and rejects only the dot segments.
18+
*
19+
* **The assertions below pin exact encoded output and exact error text on
20+
* purpose.** `safeUrlPath` lives in `tools/url-path.ts`, owned by #7262, which
21+
* this branch is rebased onto — so its behaviour changes land underneath this
22+
* file. Twice now that is precisely how a change was caught: segment trimming
23+
* being dropped, and the empty-segment check narrowing from `!segment.trim()`
24+
* to `!segment`. Rewriting these into `toThrow()` would have let both through
25+
* silently.
1826
*/
1927
import { describe, expect, it } from 'vitest'
2028
import {
2129
discoverPathParams,
2230
itPassesLegitimateValues,
2331
itResistsTraversal,
24-
toolsWithoutPathParams,
2532
} from '@/tools/__tests__/path-safety'
2633
import * as supabaseTools from '@/tools/supabase/index'
2734
import { encodeStoragePath, encodeStorageSegment } from '@/tools/supabase/utils'
@@ -72,6 +79,7 @@ const {
7279
covered: PATH_PARAMS,
7380
unbuildable: UNBUILDABLE,
7481
undiscoverable: UNDISCOVERABLE,
82+
withoutPathParams: WITHOUT_PATH_PARAMS,
7583
} = discoverPathParams(supabaseTools, 'supabase_', FIXED)
7684

7785
/**
@@ -93,7 +101,7 @@ describe('supabase path traversal safety', () => {
93101
})
94102

95103
it('leaves only genuinely static-URL tools without a path parameter', () => {
96-
expect(toolsWithoutPathParams(supabaseTools, 'supabase_', FIXED)).toEqual(STATIC_URL_TOOLS)
104+
expect(WITHOUT_PATH_PARAMS).toEqual(STATIC_URL_TOOLS)
97105
})
98106

99107
it('covers every parameter that reaches a URL path segment', () => {

0 commit comments

Comments
 (0)