Skip to content

refactor(export): use @pgpmjs/migrate-client ORM for sql_actions fetching#836

Open
pyramation wants to merge 18 commits intomainfrom
devin/1773717246-export-use-migrate-client
Open

refactor(export): use @pgpmjs/migrate-client ORM for sql_actions fetching#836
pyramation wants to merge 18 commits intomainfrom
devin/1773717246-export-use-migrate-client

Conversation

@pyramation
Copy link
Contributor

@pyramation pyramation commented Mar 17, 2026

refactor(export): add GraphQL export flow + use @pgpmjs/migrate-client ORM for sql_actions

Summary

Introduces a full GraphQL-based export flow (exportGraphQL) that mirrors the existing SQL-based exportMigrations, and replaces the hand-rolled GraphQLClient.fetchAllNodes() call for sql_actions with the typed @pgpmjs/migrate-client ORM.

Key changes:

  • export-graphql.ts: Uses createMigrateClient from @pgpmjs/migrate-client for sql_actions fetching with typed findMany() + cursor-based pagination, instead of raw GraphQL string queries
  • export-utils.ts (new): Extracts shared constants (DB_REQUIRED_EXTENSIONS, META_TABLE_ORDER, etc.) and helpers (makeReplacer, preparePackage, detectMissingModules) from export-migrations.ts so both SQL and GraphQL flows share the same logic
  • export-graphql-meta.ts (new): GraphQL-based metadata export (databases, schemas, tables, fields, services, etc.)
  • graphql-client.ts (new): Simple GraphQL HTTP client with retry logic and pagination support (still used for meta endpoint)
  • graphql-naming.ts (new): PostgreSQL ↔ PostGraphile camelCase name conversion helpers
  • export-migrations.ts: Refactored to import from export-utils.ts; adds argv parameter passthrough
  • export-parity.test.ts: Integration test for SQL vs GraphQL export parity, with mocks for both GraphQLClient and @pgpmjs/migrate-client ORM

Updates since last revision

  • Fixed CI module resolution: Changed jest.doMock for @pgpmjs/migrate-client from bare package name to absolute path (path.resolve(__dirname, '../../../sdk/migrate-client/dist/index')). The bare name failed in CI because pgpm/cli doesn't have the package as a dependency — only pgpm/core does. The absolute path pattern matches how the GraphQLClient mock already works.
  • Fixed schema prefix parity bug: The GraphQL flow's meta replacer was using metaExtensionName as the schema prefix (producing e.g. pets_export_svc_public), while the SQL flow uses extensionName (producing pets_export_public). Added schemaPrefix: name to makeReplacer() in export-graphql.ts to match the SQL flow — identical to how export-migrations.ts already does it.
  • CI is now fully green (41/41 checks pass), including the pgpm/cli export-parity integration test.

Review & Testing Checklist for Human

  • Test end-to-end with a running Constructive instance — Run pgpm export with a --migrate-endpoint pointing to a real db_migrate GraphQL API. Verify sql_actions are fetched correctly and the output matches the SQL-based flow. The test mocks the client entirely, so this is the only way to validate the ORM integration works against a real PostGraphile API.
  • Verify databaseId filter shape — Old code passed { databaseId } as a flat condition; new code uses { databaseId: { equalTo: databaseId } }. Confirm the migrate-client ORM expects PostGraphile-style filter syntax (it should, since it's codegen'd from the schema, but this has not been validated against a live server).
  • Verify cursor-based pagination with large datasets — The while (hasNextPage) loop in export-graphql.ts pages through 100 rows at a time. The test mock returns everything in one page, so multi-page behavior is untested. Confirm with a database that has >100 sql_actions.
  • Verify header merging order — Token is spread first, then migrateHeaders is spread on top. If migrateHeaders contains its own Authorization key, it will override the token-based header. Confirm this is the desired precedence.

Notes

Link to Devin Session: https://app.devin.ai/sessions/2b96fabbdaf4455fa4de78f248020902
Requested by: @pyramation

@devin-ai-integration
Copy link
Contributor

🤖 Devin AI Engineer

I'll be helping with this pull request! Here's what you should know:

✅ I will automatically:

  • Address comments on this PR. Add '(aside)' to your comment to have me ignore it.
  • Look at CI failures and help fix them

Note: I can only respond to comments from users who have write access to this repository.

⚙️ Control Options:

  • Disable automatic comment and CI monitoring

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants