Skip to content

Add an Effect-native TDS driver for sql-mssql - #8129

Open
joepjoosten wants to merge 4 commits into
Effect-TS:mainfrom
joepjoosten:feat/native-mssql-tds
Open

Add an Effect-native TDS driver for sql-mssql#8129
joepjoosten wants to merge 4 commits into
Effect-TS:mainfrom
joepjoosten:feat/native-mssql-tds

Conversation

@joepjoosten

@joepjoosten joepjoosten commented Sep 8, 2026

Copy link
Copy Markdown
Contributor

Summary

Following the native PostgreSQL driver in #7426, replace @effect/sql-mssql's
tedious runtime dependency with a native TDS 7.4 implementation owned by Effect.
Tedious remains a devDependency only for reproducible comparisons.

This is a draft, not a claim of complete tedious compatibility.

Implemented:

  • Incremental bounded packet/token parsing and native RPC/parameter codecs.
  • TLS with certificate verification by default, LOGIN7 SQL authentication,
    NTLMv2 and Security Token FedAuth, named-instance SSRP discovery, routing and
    bounded startup retries. Access tokens are acquired through an Effect for each
    new pooled connection. LOGIN7 also advertises UTF-8 support.
  • Scalar, Unicode/ANSI/collation, binary, date/time, decimal, LOB and TVP support;
    stored procedures, output parameters and return values.
  • Effect-scoped connections, existing pool integration, transactions/savepoints,
    serialized requests, timeouts and ATTENTION acknowledgement before reuse.
  • Native public MssqlTypes, preserving SQL bigint strings and numeric results.
  • Procedure calls now respect the transaction's pinned connection.

Validation

  • Full CI is green
    on a8ce97f59: Node, Bun, Deno, static checks, types, build, docs and bundle.
  • 95 tests across 11 MSSQL test files passed on Node, Bun and Deno with SQL Server
    2022 CU26 in Docker, including existing persistence/cache/queue suites.
  • The live driver suite also passes with its automatically started testcontainer,
    without any manually configured port. This fixes the initial CI connection-refused
    failures; exhaustive framing tests run serially to avoid starving sibling tests.
  • Fixed TLS record sizing on Deno and handshake framing on resumed sessions;
    the encrypted peer asserts handshake records remain TDS-wrapped through reconnects.
  • Temporal codecs preserve sub-millisecond precision and correctly round midnight
    rollover at reduced scales, verified against SQL Server.
  • Encrypted FedAuth peer tests verify token encoding, echo negotiation, mandatory
    acknowledgements, malformed acknowledgement rejection and per-connection providers.
  • Packet fragmentation/bounds, simulated routing/SSRP/NTLM, startup retries,
    cancellation races and timeout recovery tests.
  • NTLMv2 checked against published Microsoft vectors; no live domain test yet.
  • pnpm lint-fix and package TypeScript check pass.
  • Full pnpm check passes with the exact candidate in an isolated checkout.
    The nested development worktree reports unrelated test narrowing errors, so
    clean-baseline and candidate checks were repeated separately.
  • pnpm jsdocs --check reports two errors in unchanged packages/effect/src/Logger.ts
    (@see ordering and consolePretty resolution), none in this package.

Live test command:

EFFECT_INTEGRATION_TESTS=1 MSSQL_PORT=14339 pnpm test --run packages/sql/mssql/test

The test helper uses fresh scoped databases when reusing an external container.

Benchmarks

Checked-in harness, raw samples and reproduction instructions are under
packages/sql/mssql/benchmark. Same TLS server, session settings, row conversion,
result equality checks and alternating pairs against tedious 20.0.0.

Local paired median throughput changes after the TLS fixes: parameterized SELECT
+12.2%, 100×3 rowset +19.6%, 100×20 rowset +10.2%, large Unicode result +7.2%,
large Unicode parameter +10.4%, transaction cycle +11.3%.
The reused tedious DONEPROC parser workload shows +87.2% in a narrow microbenchmark.
The Docker amd64-on-ARM host was not isolated and samples contain outliers;
these are directional results, not release-level performance guarantees.

Remaining before a compatibility-complete replacement

  • Live Azure SQL verification and automatic credential flows / FEDAUTHINFO negotiation.
    Security Token FedAuth is implemented, but applications supply the token provider.
  • Live Windows-domain NTLM validation and Extended Protection/channel binding.
  • Broader interoperability/performance runs, including other server versions,
    concurrent pools and memory/GC measurements.

Streaming was not implemented by the previous adapter and remains unsupported.
Rows are buffered with a 16 MiB per-token safety limit. UDT/Variant values can
be decoded but cannot be encoded as parameters; TVP rows are synchronous arrays.
Native descriptors replace tedious descriptor objects. Compatibility notes are
in the package README and changeset.

Protocol references: MS-TDS LOGIN7
and complete authentication token state.

@changeset-bot

changeset-bot Bot commented Sep 8, 2026

Copy link
Copy Markdown

🦋 Changeset detected

Latest commit: 73abe74

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 30 packages
Name Type
@effect/sql-mssql Minor
effect Minor
@effect/opentelemetry Minor
@effect/vitest Minor
@effect/ai-anthropic Minor
@effect/ai-openai-compat Minor
@effect/ai-openai Minor
@effect/ai-openrouter Minor
@effect/atom-react Minor
@effect/atom-solid Minor
@effect/atom-vue Minor
@effect/platform-browser Minor
@effect/platform-bun Minor
@effect/platform-deno Minor
@effect/platform-node-shared Minor
@effect/platform-node Minor
@effect/sql-clickhouse Minor
@effect/sql-d1 Minor
@effect/sql-libsql Minor
@effect/sql-mysql2 Minor
@effect/sql-pg Minor
@effect/sql-pglite Minor
@effect/sql-sqlite-bun Minor
@effect/sql-sqlite-do Minor
@effect/sql-sqlite-node Minor
@effect/sql-sqlite-react-native Minor
@effect/sql-sqlite-wasm Minor
@effect/docgen Minor
@effect/doctest Minor
@effect/openapi-generator Minor

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

@github-actions

github-actions Bot commented Sep 8, 2026

Copy link
Copy Markdown
Contributor

Bundle Size Analysis

Generated from PR build output; treat the content below as untrusted.

File Name Current Size Previous Size Difference
arbitrary-combinators.ts 34.23 KB 34.23 KB 0.00 KB (0.00%)
basic.ts 6.87 KB 6.87 KB 0.00 KB (0.00%)
batching.ts 10.15 KB 10.15 KB 0.00 KB (0.00%)
brand.ts 6.45 KB 6.45 KB 0.00 KB (0.00%)
cache.ts 10.77 KB 10.77 KB 0.00 KB (0.00%)
config.ts 21.50 KB 21.50 KB 0.00 KB (0.00%)
differ.ts 20.29 KB 20.29 KB 0.00 KB (0.00%)
http-client.ts 22.11 KB 22.11 KB 0.00 KB (0.00%)
http-router.ts 33.57 KB 33.57 KB 0.00 KB (0.00%)
logger.ts 10.88 KB 10.88 KB 0.00 KB (0.00%)
metric.ts 9.02 KB 9.02 KB 0.00 KB (0.00%)
optic.ts 6.70 KB 6.70 KB 0.00 KB (0.00%)
pubsub.ts 15.27 KB 15.27 KB 0.00 KB (0.00%)
queue.ts 11.85 KB 11.85 KB 0.00 KB (0.00%)
schedule.ts 10.96 KB 10.96 KB 0.00 KB (0.00%)
schema-binary.ts 39.48 KB 39.48 KB 0.00 KB (0.00%)
schema-class.ts 20.03 KB 20.03 KB 0.00 KB (0.00%)
schema-fromJsonSchemaDocument.ts 30.42 KB 30.42 KB 0.00 KB (0.00%)
schema-representation-roundtrip.ts 26.24 KB 26.24 KB 0.00 KB (0.00%)
schema-string-transformation.ts 13.64 KB 13.64 KB 0.00 KB (0.00%)
schema-string.ts 11.12 KB 11.12 KB 0.00 KB (0.00%)
schema-template-literal.ts 15.44 KB 15.44 KB 0.00 KB (0.00%)
schema-toArbitrary.ts 33.77 KB 33.77 KB 0.00 KB (0.00%)
schema-toCodeDocument.ts 24.48 KB 24.48 KB 0.00 KB (0.00%)
schema-toCodecJson.ts 19.24 KB 19.24 KB 0.00 KB (0.00%)
schema-toEquivalence.ts 19.39 KB 19.39 KB 0.00 KB (0.00%)
schema-toFormatter.ts 19.49 KB 19.49 KB 0.00 KB (0.00%)
schema-toJsonSchemaDocument.ts 23.58 KB 23.58 KB 0.00 KB (0.00%)
schema-toRepresentation.ts 19.55 KB 19.55 KB 0.00 KB (0.00%)
schema.ts 19.26 KB 19.26 KB 0.00 KB (0.00%)
stm.ts 12.80 KB 12.80 KB 0.00 KB (0.00%)
stream.ts 9.83 KB 9.83 KB 0.00 KB (0.00%)

@joepjoosten

Copy link
Copy Markdown
Contributor Author

Benchmark results

Results for a8ce97f59, measured after the TLS framing/write-queue fixes against tedious 20.0.0.

Environment: Node 24.20.0, SQL Server 2022 CU26 (16.0.4265.3), Linux amd64 Docker container under emulation on an ARM Mac, with TLS enabled for both drivers.

Workload Native operations/s Tedious operations/s Median paired change
Parameterized SELECT 2,202 1,963 +12.2%
100 rows × 3 columns 1,114 968 +19.6%
100 rows × 20 columns 1,065 966 +10.2%
Large Unicode result 1,107 1,031 +7.2%
Large Unicode parameter 1,085 982 +10.4%
Begin / insert / rollback 358 316 +11.3%
DONEPROC parser (tokens/s) 9,540,062 5,086,195 +87.2%

Method

  • Live comparisons use the same process, TLS server, session settings, Effect callbacks and row-object conversion. Result equality is checked before timing.
  • Each live workload warms both drivers for 300 ms, then measures five alternating pairs of 750 ms. Reported changes are the median of paired throughput changes, not the ratio of independent medians.
  • The large parameter sends and returns 10,000 Unicode characters, exercising multi-packet requests. Transactions count one begin/insert/rollback cycle; native transaction control uses SQL batches, while tedious uses its transaction API.
  • The parser microbenchmark reuses the DONEPROC workload from tedious, checks token counts, and measures seven alternating pairs after warmup. Live and parser benchmarks ran sequentially.

These are directional results, not production performance guarantees. The host was not isolated, unrelated activity was not controlled, and samples vary materially. The DONEPROC result is a narrow microbenchmark, not a claim about all codecs. Latency percentiles, memory/GC, remote servers and concurrent pool load still need measurement.

Benchmark guide and reproduction commands · Raw samples

@joepjoosten

Copy link
Copy Markdown
Contributor Author

Bundle-size comparison

Measured the original tedious-based adapter (addeaea0b) against the native driver after the local MD4 implementation (73abe749e).

Bundle Old tedious driver Native driver Reduction
Driver + dependencies, minified 1,585 KB 547 KB 65.5%
Same, gzipped 470 KB 208 KB 55.9%
Including Effect, minified 1,915 KB 869 KB 54.6%
Including Effect, gzipped 573 KB 309 KB 46.1%

The native version saves approximately 1 MB of minified JavaScript. It adds only iconv-lite and its safer-buffer dependency at runtime, apart from the Effect peer dependency.

Measurement method

  • esbuild 0.28.2, identical settings for both adapters: bundled and minified ESM, tree-shaking enabled, Node platform, target Node 22, no source maps.
  • Entry point: the full public packages/sql/mssql/src/index.ts, retaining all public exports and bundling runtime dependencies. Node built-ins remain external.
  • Measured both with effect / effect/* external and with Effect bundled. Both comparisons use the same installed dependency versions and Effect source.
  • Gzip level 9; KB values above use decimal units and are rounded. These are generated JavaScript bundle sizes, not npm installation sizes or runtime memory usage. Actual application bundles depend on imports and tree-shaking.

Exact byte counts:

Configuration Old minified Native minified Old gzip Native gzip
Effect external 1,585,320 546,588 470,403 207,628
Effect bundled 1,915,395 869,421 572,968 309,051

Compatibility caveat: the old dependency bundle includes automatic Azure credential-provider functionality. The native driver delegates token acquisition to the application, so this is not a completely feature-equivalent comparison; an application adding an Azure credential SDK would add its size separately.

@joepjoosten
joepjoosten marked this pull request as ready for review September 8, 2026 14:37
@joepjoosten

Copy link
Copy Markdown
Contributor Author

Bundle-size comparison with Rolldown

Also measured the original tedious-based adapter (addeaea0b) against the native driver (73abe749e) using Rolldown 1.2.7.

Bundle Old tedious driver Native driver Reduction
Driver + dependencies, minified 1,351 KB 324 KB 76.0%
Same, gzipped 405 KB 169 KB 58.2%
Including Effect, minified 1,663 KB 635 KB 61.8%
Including Effect, gzipped 499 KB 265 KB 46.9%

Measurement method

  • Full public packages/sql/mssql/src/index.ts entry point, with public exports preserved and runtime dependencies bundled.
  • Identical Rolldown settings for both versions: Node platform, Node 22 target, ESM output, tree-shaking enabled, full minification, code splitting disabled and no source maps. Each build produced one JavaScript chunk; Node built-ins remain external.
  • Compared both with Effect external and with Effect bundled, using the same installed dependency versions and Effect source.
  • Gzip level 9. KB values are decimal and rounded. These are generated JavaScript bundle sizes, not installation size or runtime memory. Actual application bundles depend on imports and tree-shaking.

Exact byte counts:

Configuration Old minified Native minified Old gzip Native gzip
Effect external 1,350,624 324,388 405,350 169,243
Effect bundled 1,663,213 635,363 498,727 264,593

The same compatibility caveat applies: the old dependency bundle includes automatic Azure credential-provider functionality, whereas the native driver delegates token acquisition to the application. Applications adding an Azure credential SDK would add its size separately.

@joepjoosten

Copy link
Copy Markdown
Contributor Author

One question remaining: should we include a effect native character encoding/decoding functions to remove the iconv-lite dependency? i see @effect/sql-mysql2 is also using this for character encoding/decoding, so including this generally could remove the dependency when there is a native mysql2 implementation

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

4.0 enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant