refactor(export): use @pgpmjs/migrate-client ORM for sql_actions fetching#836
Open
pyramation wants to merge 18 commits intomainfrom
Open
refactor(export): use @pgpmjs/migrate-client ORM for sql_actions fetching#836pyramation wants to merge 18 commits intomainfrom
pyramation wants to merge 18 commits intomainfrom
Conversation
Contributor
🤖 Devin AI EngineerI'll be helping with this pull request! Here's what you should know: ✅ I will automatically:
Note: I can only respond to comments from users who have write access to this repository. ⚙️ Control Options:
|
Closed
6 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
refactor(export): add GraphQL export flow + use @pgpmjs/migrate-client ORM for sql_actions
Summary
Introduces a full GraphQL-based export flow (
exportGraphQL) that mirrors the existing SQL-basedexportMigrations, and replaces the hand-rolledGraphQLClient.fetchAllNodes()call forsql_actionswith the typed@pgpmjs/migrate-clientORM.Key changes:
export-graphql.ts: UsescreateMigrateClientfrom@pgpmjs/migrate-clientfor sql_actions fetching with typedfindMany()+ cursor-based pagination, instead of raw GraphQL string queriesexport-utils.ts(new): Extracts shared constants (DB_REQUIRED_EXTENSIONS,META_TABLE_ORDER, etc.) and helpers (makeReplacer,preparePackage,detectMissingModules) fromexport-migrations.tsso both SQL and GraphQL flows share the same logicexport-graphql-meta.ts(new): GraphQL-based metadata export (databases, schemas, tables, fields, services, etc.)graphql-client.ts(new): Simple GraphQL HTTP client with retry logic and pagination support (still used for meta endpoint)graphql-naming.ts(new): PostgreSQL ↔ PostGraphile camelCase name conversion helpersexport-migrations.ts: Refactored to import fromexport-utils.ts; addsargvparameter passthroughexport-parity.test.ts: Integration test for SQL vs GraphQL export parity, with mocks for bothGraphQLClientand@pgpmjs/migrate-clientORMUpdates since last revision
jest.doMockfor@pgpmjs/migrate-clientfrom bare package name to absolute path (path.resolve(__dirname, '../../../sdk/migrate-client/dist/index')). The bare name failed in CI becausepgpm/clidoesn't have the package as a dependency — onlypgpm/coredoes. The absolute path pattern matches how theGraphQLClientmock already works.metaExtensionNameas the schema prefix (producing e.g.pets_export_svc_public), while the SQL flow usesextensionName(producingpets_export_public). AddedschemaPrefix: nametomakeReplacer()inexport-graphql.tsto match the SQL flow — identical to howexport-migrations.tsalready does it.pgpm/cliexport-parity integration test.Review & Testing Checklist for Human
pgpm exportwith a--migrate-endpointpointing to a real db_migrate GraphQL API. Verify sql_actions are fetched correctly and the output matches the SQL-based flow. The test mocks the client entirely, so this is the only way to validate the ORM integration works against a real PostGraphile API.databaseIdfilter shape — Old code passed{ databaseId }as a flatcondition; new code uses{ databaseId: { equalTo: databaseId } }. Confirm the migrate-client ORM expects PostGraphile-style filter syntax (it should, since it's codegen'd from the schema, but this has not been validated against a live server).while (hasNextPage)loop inexport-graphql.tspages through 100 rows at a time. The test mock returns everything in one page, so multi-page behavior is untested. Confirm with a database that has >100 sql_actions.migrateHeadersis spread on top. IfmigrateHeaderscontains its ownAuthorizationkey, it will override the token-based header. Confirm this is the desired precedence.Notes
GraphQLClientis still used for the meta endpoint (metaschema/services data); only the migrate (sql_actions) endpoint was switched to the typed ORM@pgpmjs/migrate-clientwas added as a workspace dependency in a prior merged PR (feat: add @pgpmjs/migrate-client workspace package with zero codegen deps #834)inflektpackage was added as a dependency for camelCase ↔ snake_case conversions in the GraphQL naming helpersjest.doMockfor@pgpmjs/migrate-clientuses an absolute path tosdk/migrate-client/dist/index— if the workspace layout changes, this path will need updatingfeature/pgpm-export-graphqlbase branch)Link to Devin Session: https://app.devin.ai/sessions/2b96fabbdaf4455fa4de78f248020902
Requested by: @pyramation