postgres-backup-s3 not updated + CD pipeline #141
Comments
|
I haven't had the time to update docker hub :( |
|
@iloveitaly Are you maintaining the images on Docker Hub? As mentioned I'd be willing to set up a CD pipeline. |
|
@scriptcoded if you could set up a CD pipeline, that would be amazing. Yup I'm helping with maintenance here! |
|
@iloveitaly I'll mock something up and raise a PR. Going on vacation at the end of the week, so we'll see when I manage to get it done! :) |
|
I’ve thought a little bit about how this would best be done and have come up with three alternatives, each building upon the previous. Just for clarity, when I say “package” I’m talking about one of the directories residing in the root of this repository. Alternative 1It seems that currently all Docker Images in this repo is pushed to the Alternative 2So we have to solve the issue of maintaining multiple versions of packages in a monorepo. For tracking which version each package is on I can see three viable options:
Personally I prefer option 3 as it’s Git-native and makes it easier for the developer to push a new release as the CD flow could be triggered when a new tag is created and pushed: Regardless of the approach we would then push the following tags to Docker Hub:
By pushing different semver “levels” we allow users to receive only patch or minor updates while staying on the same major/mino version. The added amount of tags would not increase the amount of images being built or the amount of storage being used on Docker Hub (even though I’m pretty sure they don’t limit storage) as each tag is just a reference to the same image, just like Git tags. Alternative 3Having versioned, up to date images on Docker Hub is already great, but there is still one issue that persists. Some images target specific versions of software by nature. Take So ideally to solve this we would release tags like this for each image: I haven’t played around with this yet, but I’d imagine we’d have some form of manifest file in each package that needs this feature. That file would contain a list of arguments to use when building the Dockerfile. So the manifest might look something like this: {
"postgresVersion": [14, 13, 12]
}And the Dockerfile would then be built three times with that arg being passed: docker build --build-arg postgresVersion=14 .
docker build --build-arg postgresVersion=13 .
docker build --build-arg postgresVersion=12 .I’d love to hear your thoughts and discuss which direction you’d like to go. As someone using dockerfiles from this repo myself I’d very much like to see alternative 3 being implemented, but I fully understand if it adds too much complexity to the project. Cheers, |
|
Hello! Thank you for this detailed response.
I agree that option #3 is the best, but it would add additional complexity
to the project that I'm not sure is warranted given how infrequently things
are updated here (hopefully this change will help improve velocity here!).
Why don't we start with option #2. I like the third option (git tags) but
the issue I see with that approach is tags can only be managed by users who
have write access to the repo and you can't include tags in a PR. For this
reason I think a package-level versions.txt would be best. Let me know if
you see this differently, I might be missing something with what is
possible via tags.
…On Tue, Mar 22, 2022 at 5:35 AM Malcolm Nihlén ***@***.***> wrote:
@iloveitaly <https://github.com/iloveitaly>
I’ve thought a little bit about how this would best be done and have come
up with three alternatives, each building upon the previous.
Just for clarity, when I say “package” I’m talking about one of the
directories residing in the root of this repository.
Alternative 1
It seems that currently all Docker Images in this repo is pushed to the
:latest tag on Docker Hub. This would be simple to recreate in a CD
pipeline as we could just check which package have been modified in a
certain commit and push a new version of :latest for that package. While
that’s very easy to implement the big downside is that usage of :latest
is generally discouraged and makes it impossible to pin dependencies in a
production environment.
Alternative 2
So we have to solve the issue of maintaining multiple versions of packages
in a monorepo. For tracking which version each package is on I can see
three viable options:
1. Each package has a version.txt file (the name doesn’t matter of
course) whose sole purpose is to track the version of the package.
2. There is a versions.json file in the root of the project containing
the versions of all packages.
3. We track versions of each package using Git tags formatted like
postgres-backup-s3/v1.2.3. An issue with this approach is that the
latest version displayed on GitHub will always be the latest version of the
last updated package, as there is no way to display versions for multiple
tags.
Personally I prefer option 3 as it’s Git-native and makes it easier for
the developer to push a new release as the CD flow could be triggered when
a new tag is created and pushed: git tag -a postgres-backup-s3/v1.2.3 -m
"postgres-backup-s3 version 1.2.3". For options 1 and 2 we’d probably
have to check whenever the version files are modified and check if the
version has changed.
Regardless of the approach we would then push the following tags to Docker
Hub:
- postgres-backup-s3:v1.2.3
- postgres-backup-s3:v1.2
- postgres-backup-s3:v1
- postgres-backup-s3:latest
By pushing different semver “levels” we allow users to receive only patch
or minor updates while staying on the same major/mino version.
The added amount of tags would not increase the amount of images being
built or the amount of storage being used on Docker Hub (even though I’m
pretty sure they don’t limit storage) as each tag is just a reference to
the same image, just like Git tags.
Alternative 3
Having versioned, up to date images on Docker Hub is already great, but
there is still one issue that persists. Some images target specific
versions of software by nature. Take postgres-backup-s3 for example. It
uses postgresql-client whose version must align with the version of
Postgres being targeted, but to use this backup image you must just hope
that the currently released version uses the same version of
postgresql-client as your database.
So ideally to solve this we would release tags like this for each image:
postgres-backup-s3:{postgresVersion}-v{imageVersion}. This would allow
the user to specify which version of Postgres to target.
I haven’t played around with this yet, but I’d imagine we’d have some form
of manifest file in each package that needs this feature. That file would
contain a list of arguments to use when building the Dockerfile. So the
manifest might look something like this:
{
"postgresVersion": [14, 13, 12]
}
And the Dockerfile would then be built three times with that arg being
passed:
docker build --build-arg postgresVersion=14 .
docker build --build-arg postgresVersion=13 .
docker build --build-arg postgresVersion=12 .
------------------------------
I’d love to hear your thoughts and discuss which direction you’d like to
go. As someone using dockerfiles from this repo myself I’d very much like
to see alternative 3 being implemented, but I fully understand if it adds
too much complexity to the project.
Cheers,
Malcolm
—
Reply to this email directly, view it on GitHub
<#141 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AABE2RY47CXF23NUFL57ZWTVBGWAXANCNFSM5QPYKDKQ>
.
You are receiving this because you were mentioned.Message ID:
***@***.***>
|
|
@iloveitaly Hey! Just an update here. Unfortunately I've hit a bit of a snag in life, and am currently under a lot of stress. I'll get to this, I promise! It'll just have to wait a while. Have a nice weekend! |
|
No worries! I hope things get better for you soon.
…On Fri, Apr 29, 2022 at 5:58 AM Malcolm Nihlén ***@***.***> wrote:
@iloveitaly <https://github.com/iloveitaly> Hey! Just an update here.
Unfortunately I've hit a bit of a snag in life, and am currently under a
lot of stress. I'll get to this, I promise! It'll just have to wait a while.
Have a nice weekend!
—
Reply to this email directly, view it on GitHub
<#141 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AABE2R57MH6BRJJVLMLKJT3VHPFGXANCNFSM5QPYKDKQ>
.
You are receiving this because you were mentioned.Message ID:
***@***.***>
|
|
@scriptcoded @iloveitaly Any plans to get the Docker images up-to-date on Docker Hub again? |
|
Hi @tholu! I've since switched jobs and am not actively using this project anymore. If there is still a need for automation from the maintainer, reach out! But I won't be updating things manually 😊 |
Hey, nice repo!
It seems that postgres-backup-s3 is not up to date on Docker Hub. The last commit was just two months ago, but Docker Hub says "Last pushed 4 years ago". Any chance it's been missed?
Also, I saw in #57 (comment) that you were looking into setting up a CD pipeline pipeline to automatically release new versions of your images. Is this something that you've gotten around to do, or are you open to solutions? I'd be happy to take a look at it, but don't want to do double work in case it's already done 😄
Cheers!
Malcolm
The text was updated successfully, but these errors were encountered: