Stack conversion

[barrelltitor]

Proposed change

I wanted to convert stacks to git/local so I don't have to recreate them since that's painfully slow and annoying.

You can test by trying to convert your stacks between local and git. I tried this a bunch of times and it worked fine, but not sure for other complex configurations how it would be have.

Closes #636 possibly others too

Type of change

• Bug fix: non-breaking change which fixes an issue.
• New feature / Enhancement: non-breaking change which adds functionality.
• Breaking change: fix or feature that would cause existing functionality to not work as expected.
• Other. Please explain:

[CLAassistant]

All committers have signed the CLA.

[barrelltitor]

I'm not sure what you force pushed there but it seems to have broken the PR a bit, want me to rebase and redo it?

[jotka]

I'm not sure what you force pushed there but it seems to have broken the PR a bit, want me to rebase and redo it?

yes, sorry for the FP, thanks for rebasing. I will take a look, thanks!

[jotka]

Reviewed — the design is sound and the round-trip works. A few non-blocking things worth confirming, none of them data-loss:

1. Never-deployed git stack edge case. Detach relies on the compose file already living in $DATA_DIR/stacks/{env}/{stackName}/ — which is true because every git deploy copies the repo into that dir (stacks.ts cpSync(sourceDir → getStackDir())) and records stackSources.composePath there, so deleteGitStackFiles() only removes the throwaway clone under GIT_REPOS_DIR and the real compose survives. But if a git stack was created and never successfully deployed, composePath is NULL and nothing was copied yet — detaching that yields an internal stack with an .env but no compose file, so it becomes undeployable. Worth guarding: refuse detach (or warn) when the stack has no resolvable/deployed compose file.

2. if (source?.envPath !== ) guard in detach. Empty-string envPath is the convention for "no env file", but this condition also runs for NULL. It works, but please confirm the intent matches the internal-stack env convention rather than relying on the NULL-vs-empty-string distinction by accident.

3. attach + externally-edited .env. For an internal stack, non-secret vars are the .env file on disk, not necessarily all in the DB. attach carries vars via getStackEnvVars(..., false) (DB), so a stack whose .env was edited outside Dockhand (SSH/git) could drop those non-secret vars on conversion. In practice the UI keeps them in sync, so this is an edge case — just flagging it.

Thanks for this — it closes a real annoyance.

[barrelltitor]

First issue is sorted out, looks like all my testing was indeed on stacks I already deployed already, my bad

For 2 that is intentional, null and empty are different states. I don't remember exactly, but I think it was on git stacks that did not have a custom path for the .env file and just used the one in the same folder. That is probably it looking at the code for upserting stacks

For the edge case what should be the source of truth if a key value pair is changed, so e.g TZ=UTC(DB) is found in the .env file to be TZ=Europe/Bucharest, which do we apply? I'd assume the one in the .env file?

Fixing the edge case is easy, I wanted to just reload the .env file right before conversion

[barrelltitor]

Sorry for the delay, had a lot to do. So I fixed this edgecase, but I had the user be the source of truth for the actual value of the same variables and just added on newly added ones from .env

[barrelltitor]

@jotka fixed the conflicts and the edge case if you have some time to check it out again

I also built the image under ghcr.io/barrelltitor/dockhand:stack_conversion if anyone wants to try out the feature

I've been using this on my own installation and it's made my life much easier

I was also thinking to maybe have a way to PR or commit a change, but that's for another PR

[TheToto]

I tried your branch to attach my "Untracked" stack (deployed from a previous portainer instance) to Git.
It just says "Stack not found" in the "Convert to git" page.

I managed to make it work by cloning the git repo locally, "Adopt" the compose file to put the stack in the "Internal" state and then "Convert to Git".

Thank you for your work

[barrelltitor]

I tried your branch to attach my "Untracked" stack (deployed from a previous portainer instance) to Git. It just says "Stack not found" in the "Convert to git" page.

I managed to make it work by cloning the git repo locally, "Adopt" the compose file to put the stack in the "Internal" state and then "Convert to Git".

Thank you for your work

Thanks for testing, Toto!

This PR is for converting existing stacks into git/local stacks rather than adopting untracked stacks directly to git.

Are you expecting these adopted stacks to live in the portainer data directory, or in the default dockhand location?

To adopt the existing compose location you would need to first do a normal adoption to find the compose location, then convert to git, which is a better flow if you want to actually preserve the location

If you don't care about location preservation and want to basically create a new stack in place to move it to dockhand, that is a good idea and I'll add it to the PR

[barrelltitor]

I did see that the button is on untracked stacks too, that was an error on my part, didn't test with untracked stacks. Rather than removing it though I'd like to implement this as I do have a few git stacks of my own I'd like to "move over" to dockhand from portainer's messy file structure

[TheToto]

If you don't care about location preservation and want to basically create a new stack in place to move it to dockhand, that is a good idea and I'll add it to the PR

Yes that's exactly my use case :) I can do this by hand (down the stack in portainer, re-create in dockhand), but it's time saver and less error prone for a lot of stacks.

[barrelltitor]

Yeah I have the same thing, though what I did was a script to export portainer data and a little html page to see it so I can copy and paste with portainer being off and all stacks down, but it really is annoying doing it manually. Will get it done and test

[barrelltitor]

@TheToto sorry for the delay. was busy with stuff

This now works fine, I tried it myself on my own dockhand instance and I'm using it for now

You can use the image I built directly for testing ghcr.io/barrelltitor/dockhand:stack_conversion

One issue is you still need to manually enter the env variables. Does that matter?

[TheToto]

Hey,
It works fine! Thank you
The env variables thing is not really an issue.

[barrelltitor]

Rebased again as there were conflicts. @jotka when you have some time please let me know if anything else needs changed