Stack deployment from git repo with submodules

[rockaut]

Question:
Heja there,

I can't get my current setup up and running currently and I struggle to understand what I get wrong.

I have multiple public repositories - for each stack one. I also have an private repo which serves as my umbrella repository and combines all the other repositories as submodules - this one contains some secret files and such. This way I can just clone/pull my private repo and keep all the other things open. If I try to get one of this stacks up and running in portainer-ce (v2.9.1) it states that the compose file is not found.

As a test I tried to deploy the stack directly from the stack-repo which worked. And also a compose file directly from my private repo worked. So I guess your current implementation doesn't clones/pulls the submodules?
I tried to scrape through the source but my golang-fu is very week ;-)

best
Markus

[rockaut]

I found this #2669 but that isn't the case for me.

[samdulam]

@rockaut
With the current implementation it does not pull sub module content. Changed to an enhancement request.

Thanks

[rockaut]

Heja @samdulam ,

ok thank you for answering. It would be cool if there would be a way to set the git options somehow. There may be occasions where NOT pulling submodules is accepted and better.

[mariyam-portainer]

@rockaut Thanks for bringing this up, unfortunately this feature isn't currently on our roadmap right now.

[rockaut]

@mariyam-portainer you mean pulling submodules or the git options?

[huib-portainer]

What would you be trying to achieve with git options?

[rockaut]

@mariyam-portainer and @huib-portainer - sorry let's reset this: just forget about the "git options" thing, it was just an idea I mixed in. The only thing for me is the submodules support.

[bertaveira]

any plans to add this feature. We work a lot with submodules and therefore without this we can't use this git feature on portainer

[whysi]

+1 for Submodules support, from a Portainer EE customer.

It's essential for us to have this integration since almost all of our stacks are created with Git repositories having multiple compose files, using the override/extends method (explained here).
Some of those compose files are mantained by a team, some by others, in the same GIT repositories with lots of repetitions across multiple stacks.
Having the possibility to use GIT submodules will allow us to change the shared configurations in the compose files in one place, instead of committing changes in every repo, simplyfing a lot the process.

Is there any plan on adding this feature in Portainer?

[ghost]

The lack of this feature is keeping us from using repository deployment as a whole as well. I'd appreciate this feature a lot.

[bossenti]

I would really appreciate this feature as well.
Being able to use git submodules would add a lot of value to the use of the Portainer stacks for us. Hopefully, you can consider this feature for your roadmap in the near future.

[jamescarppe]

I've added some notes to our documentation that indicates that submodules aren't supported at present - this doesn't mean we're not going to implement support for this, just that we should pre-warn users of this beforehand. The feature is being tracked internally for potential future implementation.

Internal Ref: PCT-117

[insanity54]

Just adding a quick anecdote on why I want this feature. I'm trying (and failing) to use Portainer to deploy a Minetest server. Lots of mods are distributed via contentdb, Minetest's biggest mod database. These mods can be downloaded as a .zip archive and extracted to a folder. However, many mods aren't on contentdb. For one reason or another, their developer did not or does not want to go through the steps to getting the mod submitted and approved on contentdb and instead, server managers have to acquire those mods from git.

In the spirit of keeping things updated over time, I use git submodules for those mods in my minetest server repo. This way I can run git submodule update --recursive --remote and incorporate the latest changes to the mods.

Naturally, this workflow is incompatible with Portainer. A big bummer for sure. Submodules is a something I would love to see in the future. Thank you!

[LeVraiRoiDHyrule]

For me, submodules allow me to have a single .env for all my composes. My .env file contains all the passwords that are used over multiple applications, like SMTP credentials. Currently with Portainer, I need to have a .env per compose and when a password changes, I have to change it for each of my compose (18 currently) and redeploy them all. This is long and tedious. I would greatly appreciate if Portainer could simply pull submodules.

[stormsc1]

I have been following this for a while now...

Wouldn't it be possible to expose a field when creating a new stack from git, called "git args" or something along those lines? It should just be one flag when cloning or pulling if i understand this correctly?

This is one of the features that really stop us for using portainer as our primary GitOps tool. It would be awesome to see it solved.

[Schumacher-Bastian]

Allowing the use of submodules would drastically improve our workflow.

[NikitaBolshakov]

Perhaps my solution will suit you. I turn submodules into regular folders. This is a temporary solution, but it may be useful in this situation. Repo