New: support git URL/ref plugin install and source persistence#96
New: support git URL/ref plugin install and source persistence#96Copilot wants to merge 3 commits into
Conversation
taylortom
left a comment
There was a problem hiding this comment.
Good first pass — the git URL detection, schema addition, and sync path are all headed in the right direction. A few issues to fix:
Cross-PR — NO_CHANGE check in adaptframework-extras PR #10 ignores ref: That PR checks existing?.gitUrl === repository with no ref comparison. Since this PR strips the ref before storing gitUrl, changing a pinned ref in requiredPlugins config will be silently ignored on every startup. Both PRs need to store and compare gitRef — see inline comment on line 344.
CONTENTPLUGIN_ATTR_MISSING at line ~371 also carries stale name: The if (!info.targetAttribute) throw just below the diff hunk has the same issue as the inline comment on line 368 — it uses name (still pluginName, potentially '') rather than info.name. Same fix applies.
| name = pluginName | ||
| sourcePath = null | ||
| isLocalInstall = false | ||
| gitUrl = versionOrPath.split('#')[0] |
There was a problem hiding this comment.
gitRef is stripped and never stored — versionOrPath.split('#')[0] discards the branch/tag before saving. When getMissingPlugins returns p.gitUrl on a restart, the CLI installs from the default branch HEAD rather than the originally-pinned ref.
The schema needs a gitRef field alongside gitUrl, and it should be stored here:
gitUrl = versionOrPath.split('#')[0]
gitRef = versionOrPath.includes('#') ? versionOrPath.split('#')[1] : nullThen in dbData: if (isGitUrl) { dbData.gitUrl = gitUrl; dbData.gitRef = gitRef }
And getMissingPlugins should reconstruct the full URL:
if (p.gitUrl) return p.gitUrl + (p.gitRef ? `#${p.gitRef}` : '')This also fixes the NO_CHANGE detection issue in adaptframework-extras PR #10 once it compares existing?.gitRef === (ref ?? null) as well.
| isLocalInstall = false | ||
| gitUrl = versionOrPath.split('#')[0] | ||
| } else { | ||
| const pluginData = await this.findOne({ name: String(pluginName) }, { includeUpdateInfo: true, strict: false }) |
There was a problem hiding this comment.
includeUpdateInfo: true triggers an unnecessary getPluginUpdateInfos CLI subprocess — processPluginFiles only reads name and sourcePath from the result; canBeUpdated/latestCompatibleVersion are ignored entirely here. Drop the option:
const pluginData = await this.findOne({ name: String(pluginName) }, { strict: false })| const pluginData = await this.findOne({ name: String(pluginName) }, { includeUpdateInfo: true, strict: false }) | ||
| ;({ name, version, sourcePath, isLocalInstall } = await this.processPluginFiles({ ...pluginData, sourcePath: versionOrPath })) | ||
| } | ||
| const existingPlugin = await this.findOne({ name }, { strict: false }) |
There was a problem hiding this comment.
Wasted DB round-trip for git installs — when isGitUrl is true, name is pluginName (potentially ''), so findOne({ name: '' }) always returns null and the existingPlugin guard below is always skipped. Move this call inside the else branch so it only runs for non-git installs:
} else {
const pluginData = await this.findOne(...)
;({ name, version, sourcePath, isLocalInstall } = await this.processPluginFiles(...))
}
const existingPlugin = isGitUrl ? null : await this.findOne({ name }, { strict: false })| }) | ||
| } | ||
| if (isGitUrl) dbData.gitUrl = gitUrl | ||
| const info = await this.insertOrUpdate(dbData) |
There was a problem hiding this comment.
insertOrUpdate runs before isInstallSuccessful is checked — a failed git install (network error, bad ref, auth failure) writes a DB record with gitUrl set, then throws. On the next startup getMissingPlugins finds the record, returns the gitUrl, the CLI fails again, and the cycle repeats with no recovery path.
Consider only writing to the DB after confirming success:
const [data] = await this.framework.runCliCommand('installPlugins', { plugins: [pluginArg] })
if (!data.isInstallSuccessful) {
throw this.app.errors.CONTENTPLUGIN_CLI_INSTALL_FAILED.setData({ name: data.name ?? pluginName })
}
const dbData = { ...(await data.getInfo()), type: await data.getType(), isLocalInstall }
if (isGitUrl) dbData.gitUrl = gitUrl
const info = await this.insertOrUpdate(dbData)(This is a pre-existing ordering issue for all install paths, but the retry loop is a much more realistic failure mode for git installs.)
| const info = await this.insertOrUpdate(dbData) | ||
| if (!data.isInstallSuccessful) { | ||
| throw this.app.errors.CONTENTPLUGIN_CLI_INSTALL_FAILED | ||
| .setData({ name }) |
There was a problem hiding this comment.
Stale name in error payload — for git URL installs where pluginName is '', name is never updated from data.getInfo(). This error fires with { name: '' } instead of the real plugin name. Same issue applies to the CONTENTPLUGIN_ATTR_MISSING throw a few lines below.
Fix: use data.name (available after the CLI call) or info.name (available after insertOrUpdate):
throw this.app.errors.CONTENTPLUGIN_CLI_INSTALL_FAILED.setData({ name: data.name || pluginName })| "description": "Whether the plugin has been installed locally (as opposed to with the CLI)", | ||
| "type": "boolean" | ||
| }, | ||
| "gitUrl": { |
There was a problem hiding this comment.
Add "format": "uri" to match the repository field in the adaptframework-extras schema and reject malformed values before they reach the CLI:
"gitUrl": {
"description": "The HTTPS git URL this plugin was installed from, if applicable",
"type": "string",
"format": "uri"
}| }) | ||
| assert.equal(processPluginFiles.mock.callCount(), 0) | ||
| assert.equal(insertOrUpdate.mock.calls[0].arguments[0].gitUrl, 'https://github.com/org/adapt-hotgrid.git') | ||
| assert.equal(result.name, 'adapt-hotgrid') |
There was a problem hiding this comment.
Error paths are not covered — this test only exercises the happy path (isInstallSuccessful: true, targetAttribute present). Neither CONTENTPLUGIN_CLI_INSTALL_FAILED nor CONTENTPLUGIN_ATTR_MISSING is tested, so the stale-name bug (see inline comment on line 368) would not be caught by CI.
Consider adding two cases:
isInstallSuccessful: false→ assert the thrown error carries the real plugin name (not'')targetAttributemissing fromgetInfo()→ same assertion
Git-installed plugins were not fully supported in the contentplugin install/sync lifecycle, causing install failures, ref loss after restart, and stale error metadata. This change adds first-class handling for HTTPS git plugin sources including pinned refs across install, DB persistence, and missing-plugin recovery paths.
Fix
gitUrlURI validation and a newgitReffield to the contentplugin schema so git source metadata can be stored and reused safely.installPluginto parse/persist bothgitUrlandgitReffor HTTPS git installs.getMissingPluginsto reinstall non-local git plugins from reconstructedgitUrl#gitRefwhen available.syncPluginDatato persist bothgitUrlandgitReffor git sources returned bygetPluginUpdateInfos.Update
includeUpdateInfousage in the non-gitfindOnepath during install.gitRefpersistence,gitUrl#gitRef,gitUrlandgitRef,targetAttributeerror payload name resolution.New
versionpayload now accepts HTTPS git URLs with pinned refs end-to-end.Breaking
Testing
tests/ContentPluginModule.spec.jsfor git URL/ref install, sync, missing-plugin reconstruction, and error paths.node --test tests/ContentPluginModule.spec.js.npm test.npx standard.