fix(file): locate regex matches in PostgreSQL, never in JavaScript

Preview rendering ran the user's compiled pattern with `RegExp.exec` to centre
the excerpt on the match. `RegExp` matches by backtracking, and the literal-run
gate admits nested quantifiers, so `(a+)+bcd` against a long segment cost 768ms
at 40 leading `a`s and doubles with each one — synchronously, on the event loop,
once per returned row, and entirely outside the statement timeout that bounds
the query which found the row.

PostgreSQL runs that same pattern in 0.49ms: its engine does not backtrack, and
`regexp_instr` runs inside the read's transaction, so locating a match can never
cost more than having found it. Regex mode now selects the match offsets
alongside the row and `findMatchRange` returns null for it, which is the
interface's contract rather than an omission. Exact mode is unchanged — scanning
for a known string is linear.

PostgreSQL counts characters where JavaScript slices by UTF-16 unit, so the
offsets are converted by walking the segment rather than assuming either width.

Also fixes two audit failures: `getErrorMessage` in place of a hand-written
`instanceof Error` ternary, and regenerated tool metadata and integration docs
for the search params.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

Author: icecrasher321

apps/docs/content/docs/integrations/file.mdx

@@ -70,13 +70,14 @@ Extract the text content of one or more workspace files from selected file objec
 
 ### File Search
 
-Search indexed text across active workspace files using literal smart-case substring matching.
+Search every active workspace file for lines matching a regular expression, and return each match with its file ID and line number.
 
 #### Input
 
 | Parameter | Type | Required | Description |
 | --------- | ---- | -------- | ----------- |
-| `query` | string | Yes | Literal text to find \(3-512 characters\). Uppercase Unicode letters make matching case-sensitive. |
+| `query` | string | Yes | A regular expression matched against each line, 3-512 characters. Supports "." "*" "+" "?" "\{n,m\}" and their lazy forms, character classes such as "\[a-z\]" and "\[^0-9\]", the classes \d \w \s and \D \W \S, alternation "\|", groups "\(...\)" and "\(?:...\)", the anchors "^" and "$", and the word boundary \b. Lookahead, lookbehind, backreferences, named groups, inline flags such as "\(?i\)", \p\{...\} and POSIX "\[\[:alpha:\]\]" classes are not supported, and a pattern cannot span a line break. The pattern must contain at least 3 consecutive literal characters that every match will include — write "error \d+" rather than "\w+ \d+". Escape any metacharacter you mean literally. Matching is case-insensitive until the pattern contains an uppercase letter, which makes it case-sensitive. |
+| `mode` | string | No | How the query is read, chosen by the workflow builder: "regex" \(default\) as a regular expression, or "exact" as verbatim text. |
 | `maxResults` | number | No | Hard result cap configured by the workflow builder \(1-200, default 50\). |
 
 #### Output

apps/sim/lib/workspace-files/search/pattern.test.ts

@@ -79,28 +79,28 @@ describe('compileFileSearchPattern', () => {
       expect(() => compileFileSearchPattern('error \\d+', 'regex')).not.toThrow()
     })
 
-    it('locates the match so a long line previews around it', () => {
+    it('never runs the pattern against a segment in JavaScript', () => {
       const pattern = compileFileSearchPattern('needle\\d+', 'regex')
 
-      expect(pattern.findMatchRange('xxx needle42 yyy')).toEqual({ start: 4, end: 12 })
-      expect(pattern.findMatchRange('no match here')).toBeNull()
+      expect(pattern.findMatchRange('xxx needle42 yyy')).toBeNull()
     })
 
-    it('never returns a range that splits a surrogate pair', () => {
-      const pattern = compileFileSearchPattern('.needle', 'regex')
-      const line = `a🙂needle`
-      const range = pattern.findMatchRange(line)
-
-      expect(range).not.toBeNull()
-      expect([...line.slice(range?.start, range?.end)].join('')).not.toContain('�')
-      expect(line.slice(range?.start, range?.end)).toBe('🙂needle')
+    /**
+     * `RegExp` matches by backtracking, so admitting this pattern and running it
+     * here would cost seconds on one long segment — growing exponentially with
+     * the run of `a`s, on the event loop, once per returned row.
+     */
+    it('cannot be driven into backtracking by an admitted pattern', () => {
+      const pattern = compileFileSearchPattern('(a+)+bcd', 'regex')
+      const hostile = `${'a'.repeat(60)}X${'z'.repeat(3000)}bcd`
+
+      const started = performance.now()
+      expect(pattern.findMatchRange(hostile)).toBeNull()
+      expect(performance.now() - started).toBeLessThan(50)
     })
 
-    it('does not carry match state between calls', () => {
-      const pattern = compileFileSearchPattern('needle', 'regex')
-
-      expect(pattern.findMatchRange('a needle')).toEqual({ start: 2, end: 8 })
-      expect(pattern.findMatchRange('a needle')).toEqual({ start: 2, end: 8 })
+    it('still rejects a pattern that does not compile', () => {
+      expect(() => compileFileSearchPattern('needle(', 'regex')).toThrow(FileSearchPatternError)
     })
   })
 })

apps/sim/lib/workspace-files/search/pattern.ts

@@ -1,3 +1,4 @@
+import { getErrorMessage } from '@sim/utils/errors'
 import {
   FILE_SEARCH_MAX_QUERY_LENGTH,
   FILE_SEARCH_MIN_QUERY_LENGTH,
@@ -47,7 +48,15 @@ export interface CompiledFileSearchPattern {
    * to unsplit lines trades those matches for never reporting a false one.
    */
   wholeLineOnly: boolean
-  /** Locates the match inside a segment PostgreSQL already matched. */
+  /**
+   * Locates the match inside a segment PostgreSQL already matched — but only
+   * where locating it is bounded work. Exact mode scans for a known string.
+   * Regex mode returns `null`: JavaScript matches by backtracking, and an
+   * admitted pattern like `(a+)+bcd` costs seconds on one long segment and
+   * grows exponentially, so a caller that needs a regex match located asks
+   * PostgreSQL, whose engine does not backtrack and whose work is already
+   * bounded by the read's statement timeout.
+   */
   findMatchRange(segment: string): FileSearchMatchRange | null
 }
 
@@ -100,7 +109,7 @@ function findLiteralMatchRange(line: string, query: string, caseSensitive: boole
 }
 
 /** Pulls a range off a surrogate pair, so slicing it never yields a lone half. */
-function alignToCodePoints(line: string, range: FileSearchMatchRange): FileSearchMatchRange {
+export function alignToCodePoints(line: string, range: FileSearchMatchRange): FileSearchMatchRange {
   let { start, end } = range
   const startUnit = line.charCodeAt(start)
   if (start > 0 && startUnit >= 0xdc00 && startUnit <= 0xdfff) start -= 1
@@ -131,16 +140,17 @@ function compileRegexPattern(query: string): CompiledFileSearchPattern {
   const caseSensitive = isFileSearchCaseSensitive(analysis.literals)
 
   /**
-   * The subset is chosen so the same source compiles in both engines, and this
-   * is where that holds: PostgreSQL selects the rows, and this expression finds
-   * the match inside one to centre the preview on.
+   * Compiled, never executed. The subset is the intersection of the two engines,
+   * so this rejects a malformed pattern with a precise message before it costs a
+   * database round trip — but running it is what the interface's `findMatchRange`
+   * contract refuses, because compiling a regex is linear and matching with one
+   * is not.
    */
-  let matcher: RegExp
   try {
-    matcher = new RegExp(query, caseSensitive ? '' : 'i')
+    new RegExp(query, caseSensitive ? '' : 'i')
   } catch (error) {
     throw new FileSearchPatternError(
-      `Invalid search pattern: ${error instanceof Error ? error.message : 'could not be compiled'}`
+      `Invalid search pattern: ${getErrorMessage(error, 'could not be compiled')}`
     )
   }
 
@@ -150,14 +160,7 @@ function compileRegexPattern(query: string): CompiledFileSearchPattern {
     sqlPattern: analysis.postgresSource,
     literalText: null,
     wholeLineOnly: analysis.anchored,
-    findMatchRange: (segment) => {
-      const match = matcher.exec(segment)
-      if (!match) return null
-      return alignToCodePoints(segment, {
-        start: match.index,
-        end: match.index + match[0].length,
-      })
-    },
+    findMatchRange: () => null,
   }
 }
 

apps/sim/lib/workspace-files/search/repository.ts

@@ -12,7 +12,9 @@ import {
   FILE_SEARCH_STATEMENT_TIMEOUT_MS,
 } from '@/lib/workspace-files/search/constants'
 import {
+  alignToCodePoints,
   type CompiledFileSearchPattern,
+  type FileSearchMatchRange,
   FileSearchPatternError,
 } from '@/lib/workspace-files/search/pattern'
 import { createFileSearchPreview } from '@/lib/workspace-files/search/text'
@@ -105,6 +107,55 @@ function buildMatchExpression(content: SegmentContent, pattern: CompiledFileSear
     : sql`${content} ILIKE ${pattern.sqlPattern} ESCAPE '\\'`
 }
 
+/**
+ * Where the match sits inside the segment, located by PostgreSQL.
+ *
+ * A regex is never run against a segment in JavaScript: `RegExp` matches by
+ * backtracking, so an admitted pattern like `(a+)+bcd` takes seconds on one long
+ * segment and grows exponentially with it, on the event loop, once per returned
+ * row. PostgreSQL's engine does not backtrack — the same pattern resolves in
+ * under a millisecond — and this runs inside the read's statement timeout, so
+ * the cost of locating a match can never exceed the cost of having found it.
+ *
+ * Exact mode locates its own match in JavaScript, where scanning for a known
+ * string is linear, so it selects a constant here rather than paying for a
+ * second pass.
+ */
+function buildMatchOffsets(content: SegmentContent, pattern: CompiledFileSearchPattern) {
+  if (pattern.mode !== 'regex') {
+    return { matchStart: sql<number>`0`, matchEnd: sql<number>`0` }
+  }
+  const flags = pattern.caseSensitive ? '' : 'i'
+  return {
+    matchStart: sql<number>`regexp_instr(${content}, ${pattern.sqlPattern}, 1, 1, 0, ${flags})`,
+    matchEnd: sql<number>`regexp_instr(${content}, ${pattern.sqlPattern}, 1, 1, 1, ${flags})`,
+  }
+}
+
+/**
+ * PostgreSQL counts characters and JavaScript slices by UTF-16 unit, so an
+ * astral character shifts every offset after it by one. Walking the segment
+ * converts between them without assuming either width.
+ */
+function toSegmentRange(
+  content: string,
+  matchStart: number,
+  matchEnd: number
+): FileSearchMatchRange | null {
+  if (matchStart < 1 || matchEnd <= matchStart) return null
+  let units = 0
+  let characters = 0
+  let start = -1
+  while (units < content.length) {
+    if (characters === matchStart - 1) start = units
+    if (characters === matchEnd - 1) break
+    units += (content.codePointAt(units) ?? 0) > 0xffff ? 2 : 1
+    characters += 1
+  }
+  if (start < 0) return null
+  return alignToCodePoints(content, { start, end: units })
+}
+
 /** How much of the logical line surrounds a literal match, in the narrower direction. */
 function buildSurroundingContext(
   content: SegmentContent,
@@ -130,6 +181,7 @@ export async function searchWorkspaceFileIndex({
 
   const content = workspaceFileSearchSegment.content
   const matchExpression = buildMatchExpression(content, pattern)
+  const { matchStart, matchEnd } = buildMatchOffsets(content, pattern)
 
   /**
    * A logical line longer than {@link FILE_SEARCH_SEGMENT_CHARS} is stored as
@@ -185,6 +237,8 @@ export async function searchWorkspaceFileIndex({
             segmentStart: workspaceFileSearchSegment.segmentStart,
             lineLength: workspaceFileSearchSegment.lineLength,
             content: workspaceFileSearchSegment.content,
+            matchStart,
+            matchEnd,
           }
         )
         .from(workspaceFileSearchSegment)
@@ -274,6 +328,10 @@ export async function searchWorkspaceFileIndex({
       text: createFileSearchPreview(row.content, pattern, undefined, {
         prefixOmitted: row.segmentStart > 0,
         suffixOmitted: row.segmentStart + row.content.length < row.lineLength,
+        matchRange:
+          pattern.mode === 'regex'
+            ? toSegmentRange(row.content, row.matchStart, row.matchEnd)
+            : undefined,
       }),
     }))
     signal?.throwIfAborted()

apps/sim/lib/workspace-files/search/text.ts

@@ -4,7 +4,10 @@ import {
   FILE_SEARCH_SEGMENT_CHARS,
   FILE_SEARCH_SEGMENT_OVERLAP_CHARS,
 } from '@/lib/workspace-files/search/constants'
-import type { CompiledFileSearchPattern } from '@/lib/workspace-files/search/pattern'
+import type {
+  CompiledFileSearchPattern,
+  FileSearchMatchRange,
+} from '@/lib/workspace-files/search/pattern'
 
 export interface LogicalLine {
   lineNumber: number
@@ -106,17 +109,21 @@ export function truncateUtf8ToBytes(text: string, maxBytes: number): string {
 /**
  * Renders one matching segment as a bounded, match-centred excerpt.
  *
- * The pattern locates the match so the excerpt is cut around it rather than at
- * the head of the line. A pattern that PostgreSQL matched but that finds no
- * range here — the narrow cases where PostgreSQL's and JavaScript's character
- * classes disagree, such as a non-ASCII digit under `\d` — still renders, just
+ * The excerpt is cut around the match rather than at the head of the line.
+ * `matchRange` carries a match the caller already located — which is how a
+ * regex match arrives, since only PostgreSQL may run one — and otherwise the
+ * pattern locates its own. A match that neither can place still renders,
  * anchored at the start of the segment.
  */
 export function createFileSearchPreview(
   line: string,
   pattern: CompiledFileSearchPattern,
   maxBytes = FILE_SEARCH_MAX_PREVIEW_BYTES,
-  boundaries: { prefixOmitted?: boolean; suffixOmitted?: boolean } = {}
+  boundaries: {
+    prefixOmitted?: boolean
+    suffixOmitted?: boolean
+    matchRange?: FileSearchMatchRange | null
+  } = {}
 ): string {
   const boundaryBytes =
     (boundaries.prefixOmitted ? Buffer.byteLength('…', 'utf8') : 0) +
@@ -125,7 +132,8 @@ export function createFileSearchPreview(
     return `${boundaries.prefixOmitted ? '…' : ''}${line}${boundaries.suffixOmitted ? '…' : ''}`
   }
 
-  const { start: matchStart, end: matchEnd } = pattern.findMatchRange(line) ?? { start: 0, end: 0 }
+  const { start: matchStart, end: matchEnd } = boundaries.matchRange ??
+    pattern.findMatchRange(line) ?? { start: 0, end: 0 }
   const leadingEllipsis = boundaries.prefixOmitted || matchStart > 0 ? '…' : ''
   const trailingEllipsis = boundaries.suffixOmitted || matchEnd < line.length ? '…' : ''
   const ellipsisBytes = Buffer.byteLength(leadingEllipsis + trailingEllipsis, 'utf8')