Skip to content

Memory leak: functions passed to narrow, pipe, match and defaults are never released #1676

Description

@VanTanev

Follow-up to #1584 and #1673.

#1673 fixed the registry growth for repeated equivalent types.
Types that take a function still leak.

🔎 Search Terms

memory leak, OOM, narrow, pipe, match, default

🧩 Context

  • ArkType version: 2.2.7
  • TypeScript version (5.1+): 7.0.2
  • Other context you think may be relevant (JS flavor, OS, etc.): Node 24.21.0, Linux x64

🧑‍💻 Repro

Each call of handleRequest leaks memory:

import { match, type } from "arktype"

const handleRequest = () => {
	// Odd type from the docs
	const Odd = type("number").narrow((n, ctx) =>
		// if even, add a customizable error and return false
		n % 2 === 0 ? ctx.mustBe("odd") : true
	)

	// naive JSON parser from the docs
	const parseJson = type("string").pipe((s): object => JSON.parse(s))

	// sizeOf matcher from the docs
	const sizeOf = match({
		"string | Array": v => v.length,
		number: v => v,
		bigint: v => v,
		default: "assert"
	})

	// defaultable tuple from the docs
	const MyTuple = type([
		"string",
		[
			{
				name: "string"
			},
			"=",
			() => ({ name: "Anon Eemuss" })
		]
	])
}

Each call makes new functions.
ArkType never releases them.
register() keeps each function on $ark and in namesByResolution.
Each function also makes new nodes.
nodesByRegisteredId, the scope's nodesByHash, the intersection cache and the default-value caches keep these nodes.

Growth per call of each definition, over 25,000 calls with forced GC:

Definition Heap $ark entries nodesByRegisteredId entries
Odd 13.7 KB 2 3
parseJson 5.8 KB 1 1
sizeOf 70.4 KB 3 8
MyTuple 29.7 KB 4 3

Code must build a type per request when its function uses request data, for example .narrow(v => v.id === requestedId).

The test from #1673 uses no functions, so it does not catch this.
These tests fail on main (1ee4923).
They check that GC collects the Type.
They do not depend on how a fix stores the function:

import { attest, contextualize } from "@ark/attest"
import { match, type } from "arktype"
import { setFlagsFromString } from "node:v8"
import { runInNewContext } from "node:vm"

// WeakRef is not in the ES2020 lib, but node has it
declare const WeakRef: new <t extends object>(
	target: t
) => { deref(): t | undefined }

contextualize(() => {
	setFlagsFromString("--expose-gc")
	const gc: () => void = runInNewContext("gc")

	// create returns a Type that nothing else references
	const isCollected = async (create: () => object) => {
		create()
		const ref = new WeakRef(create())
		for (let i = 0; i < 3; i++) {
			await new Promise(resolve => setImmediate(resolve))
			gc()
		}
		return ref.deref() === undefined
	}

	it("narrow", async () => {
		const collected = await isCollected(() => {
			const Odd = type("number").narrow((n, ctx) =>
				// if even, add a customizable error and return false
				n % 2 === 0 ? ctx.mustBe("odd") : true
			)
			Odd(7)
			return Odd
		})
		attest(collected).equals(true)
	})

	it("pipe", async () => {
		const collected = await isCollected(() => {
			const parseJson = type("string").pipe((s): object => JSON.parse(s))
			parseJson('{ "ark": "type" }')
			return parseJson
		})
		attest(collected).equals(true)
	})

	it("match", async () => {
		const collected = await isCollected(() => {
			const sizeOf = match({
				"string | Array": v => v.length,
				number: v => v,
				bigint: v => v,
				default: "assert"
			})
			sizeOf("abc")
			return sizeOf
		})
		attest(collected).equals(true)
	})

	it("default", async () => {
		const collected = await isCollected(() => {
			const MyTuple = type([
				"string",
				[
					{
						name: "string"
					},
					"=",
					() => ({ name: "Anon Eemuss" })
				]
			])
			MyTuple(["a"])
			return MyTuple
		})
		attest(collected).equals(true)
	})
})

scope() also leaks.
Each call of scope({ a: { a: "string" } }) adds 2 entries to nodesByRegisteredId.
With .export(), each call adds 6.

Possible fix

We have a fix that passes these tests and the full test suite:

createNode(definition with fn)
  name = nameOf(fn)              // WeakMap fn -> name, no $ark write
  node.values[name] = fn         // not $ark[name] = fn
  cache node under WeakMap(fn)   // not the global nodesByHash
compile(node)
  $ark = Object.create($ark) + node.values

Nothing global keeps fn.
GC collects it with the Type, its nodes and its compiled code.
The fix changes 11 files (+284/−93).
scope().export() keeps more memory per call: 329 KB, against 176 KB on main.

Is a vibe-coded PR welcome, or is it best avoided?

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions