Skip to content

New: support git URL/ref plugin install and source persistence#96

Open
Copilot wants to merge 3 commits into
masterfrom
copilot/add-support-for-git-urls
Open

New: support git URL/ref plugin install and source persistence#96
Copilot wants to merge 3 commits into
masterfrom
copilot/add-support-for-git-urls

Conversation

Copilot AI commented Jun 6, 2026

Copy link
Copy Markdown
Contributor

Git-installed plugins were not fully supported in the contentplugin install/sync lifecycle, causing install failures, ref loss after restart, and stale error metadata. This change adds first-class handling for HTTPS git plugin sources including pinned refs across install, DB persistence, and missing-plugin recovery paths.

Fix

  • Added gitUrl URI validation and a new gitRef field to the contentplugin schema so git source metadata can be stored and reused safely.
  • Updated installPlugin to parse/persist both gitUrl and gitRef for HTTPS git installs.
  • Updated getMissingPlugins to reinstall non-local git plugins from reconstructed gitUrl#gitRef when available.
  • Updated syncPluginData to persist both gitUrl and gitRef for git sources returned by getPluginUpdateInfos.
  • Moved DB persistence to run only after successful CLI install to avoid persisting failed installs.
  • Fixed install error payload names to use resolved plugin names for both CLI-install-failed and missing-target-attribute errors.

Update

  • Removed unnecessary includeUpdateInfo usage in the non-git findOne path during install.
  • Avoided unnecessary existing-plugin lookup for git URL installs.
  • Expanded module tests to cover:
    • git URL install path with gitRef persistence,
    • missing-plugin reconstruction with gitUrl#gitRef,
    • sync persistence of gitUrl and gitRef,
    • CLI install failure error payload name resolution,
    • missing targetAttribute error payload name resolution.
  • Example install path now supported:
await installPlugin('', 'https://github.com/org/plugin.git#v2.0.0')
// CLI receives full URL; DB persists gitUrl + gitRef

New

  • No additional user-facing API surface; existing version payload now accepts HTTPS git URLs with pinned refs end-to-end.

Breaking

  • None.

Testing

  1. Added/updated focused unit tests in tests/ContentPluginModule.spec.js for git URL/ref install, sync, missing-plugin reconstruction, and error paths.
  2. Ran node --test tests/ContentPluginModule.spec.js.
  3. Ran full suite with npm test.
  4. Ran lint with npx standard.

Copilot AI changed the title [WIP] Add support for installing/updating plugins from git URLs feat: support git URL plugin install and plugin source persistence Jun 6, 2026
Copilot AI requested a review from taylortom June 6, 2026 13:40
@taylortom taylortom changed the title feat: support git URL plugin install and plugin source persistence New: support git URL plugin install and plugin source persistence Jun 6, 2026

@taylortom taylortom left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good first pass — the git URL detection, schema addition, and sync path are all headed in the right direction. A few issues to fix:

Cross-PR — NO_CHANGE check in adaptframework-extras PR #10 ignores ref: That PR checks existing?.gitUrl === repository with no ref comparison. Since this PR strips the ref before storing gitUrl, changing a pinned ref in requiredPlugins config will be silently ignored on every startup. Both PRs need to store and compare gitRef — see inline comment on line 344.

CONTENTPLUGIN_ATTR_MISSING at line ~371 also carries stale name: The if (!info.targetAttribute) throw just below the diff hunk has the same issue as the inline comment on line 368 — it uses name (still pluginName, potentially '') rather than info.name. Same fix applies.

name = pluginName
sourcePath = null
isLocalInstall = false
gitUrl = versionOrPath.split('#')[0]

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

gitRef is stripped and never storedversionOrPath.split('#')[0] discards the branch/tag before saving. When getMissingPlugins returns p.gitUrl on a restart, the CLI installs from the default branch HEAD rather than the originally-pinned ref.

The schema needs a gitRef field alongside gitUrl, and it should be stored here:

gitUrl = versionOrPath.split('#')[0]
gitRef = versionOrPath.includes('#') ? versionOrPath.split('#')[1] : null

Then in dbData: if (isGitUrl) { dbData.gitUrl = gitUrl; dbData.gitRef = gitRef }

And getMissingPlugins should reconstruct the full URL:

if (p.gitUrl) return p.gitUrl + (p.gitRef ? `#${p.gitRef}` : '')

This also fixes the NO_CHANGE detection issue in adaptframework-extras PR #10 once it compares existing?.gitRef === (ref ?? null) as well.

Comment thread lib/ContentPluginModule.js Outdated
isLocalInstall = false
gitUrl = versionOrPath.split('#')[0]
} else {
const pluginData = await this.findOne({ name: String(pluginName) }, { includeUpdateInfo: true, strict: false })

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

includeUpdateInfo: true triggers an unnecessary getPluginUpdateInfos CLI subprocessprocessPluginFiles only reads name and sourcePath from the result; canBeUpdated/latestCompatibleVersion are ignored entirely here. Drop the option:

const pluginData = await this.findOne({ name: String(pluginName) }, { strict: false })

Comment thread lib/ContentPluginModule.js Outdated
const pluginData = await this.findOne({ name: String(pluginName) }, { includeUpdateInfo: true, strict: false })
;({ name, version, sourcePath, isLocalInstall } = await this.processPluginFiles({ ...pluginData, sourcePath: versionOrPath }))
}
const existingPlugin = await this.findOne({ name }, { strict: false })

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wasted DB round-trip for git installs — when isGitUrl is true, name is pluginName (potentially ''), so findOne({ name: '' }) always returns null and the existingPlugin guard below is always skipped. Move this call inside the else branch so it only runs for non-git installs:

} else {
  const pluginData = await this.findOne(...)
  ;({ name, version, sourcePath, isLocalInstall } = await this.processPluginFiles(...))
}
const existingPlugin = isGitUrl ? null : await this.findOne({ name }, { strict: false })

Comment thread lib/ContentPluginModule.js Outdated
})
}
if (isGitUrl) dbData.gitUrl = gitUrl
const info = await this.insertOrUpdate(dbData)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

insertOrUpdate runs before isInstallSuccessful is checked — a failed git install (network error, bad ref, auth failure) writes a DB record with gitUrl set, then throws. On the next startup getMissingPlugins finds the record, returns the gitUrl, the CLI fails again, and the cycle repeats with no recovery path.

Consider only writing to the DB after confirming success:

const [data] = await this.framework.runCliCommand('installPlugins', { plugins: [pluginArg] })
if (!data.isInstallSuccessful) {
  throw this.app.errors.CONTENTPLUGIN_CLI_INSTALL_FAILED.setData({ name: data.name ?? pluginName })
}
const dbData = { ...(await data.getInfo()), type: await data.getType(), isLocalInstall }
if (isGitUrl) dbData.gitUrl = gitUrl
const info = await this.insertOrUpdate(dbData)

(This is a pre-existing ordering issue for all install paths, but the retry loop is a much more realistic failure mode for git installs.)

Comment thread lib/ContentPluginModule.js Outdated
const info = await this.insertOrUpdate(dbData)
if (!data.isInstallSuccessful) {
throw this.app.errors.CONTENTPLUGIN_CLI_INSTALL_FAILED
.setData({ name })

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Stale name in error payload — for git URL installs where pluginName is '', name is never updated from data.getInfo(). This error fires with { name: '' } instead of the real plugin name. Same issue applies to the CONTENTPLUGIN_ATTR_MISSING throw a few lines below.

Fix: use data.name (available after the CLI call) or info.name (available after insertOrUpdate):

throw this.app.errors.CONTENTPLUGIN_CLI_INSTALL_FAILED.setData({ name: data.name || pluginName })

"description": "Whether the plugin has been installed locally (as opposed to with the CLI)",
"type": "boolean"
},
"gitUrl": {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Add "format": "uri" to match the repository field in the adaptframework-extras schema and reject malformed values before they reach the CLI:

"gitUrl": {
  "description": "The HTTPS git URL this plugin was installed from, if applicable",
  "type": "string",
  "format": "uri"
}

})
assert.equal(processPluginFiles.mock.callCount(), 0)
assert.equal(insertOrUpdate.mock.calls[0].arguments[0].gitUrl, 'https://github.com/org/adapt-hotgrid.git')
assert.equal(result.name, 'adapt-hotgrid')

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Error paths are not covered — this test only exercises the happy path (isInstallSuccessful: true, targetAttribute present). Neither CONTENTPLUGIN_CLI_INSTALL_FAILED nor CONTENTPLUGIN_ATTR_MISSING is tested, so the stale-name bug (see inline comment on line 368) would not be caught by CI.

Consider adding two cases:

  1. isInstallSuccessful: false → assert the thrown error carries the real plugin name (not '')
  2. targetAttribute missing from getInfo() → same assertion

@taylortom taylortom marked this pull request as ready for review June 7, 2026 11:28
Copilot AI changed the title New: support git URL plugin install and plugin source persistence New: support git URL/ref plugin install and source persistence Jun 7, 2026
Copilot AI requested a review from taylortom June 7, 2026 11:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add support for installing/updating plugins from git URLs

2 participants