-
Notifications
You must be signed in to change notification settings - Fork 47
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Provide versioning of used images #145
Conversation
This seems like a good way to provide versioning. Do we want to keep a collection of environment files for different releases in the repo? I could see something like |
(I should say - do we plan to keep environment files in the future, since right now we don't 100% support any specific OpenStack versions) |
@timothyb89 unfortunately there's no -e for the About lack of full support, we (Fujitsu) want to have |
@timothyb89 looking at this change, I don't see anything that could be improved much further. |
Hmm, I didn't realize the environment files worked that way for docker-compose, that's unfortunate. So if you wanted to use a specific version (assuming
This would provide versioning for all images but I wonder if we could run into trouble if/when we add new containers, e.g. new init jobs or move to clustered kafka, etc. Since only image versions are read from the environment we could still break "stable" deployments in many cases. Would it make sense to version docker-compose.yml as well? Either on a git branch (like OpenStack releases) or just a subdirectory? Something like That said, if your needs are met with the single .env file I don't have any objections. |
Having branches, etc etc...would make sense only if each image that is part of this project was in its own external repo (I guess) that would be versioned in similar manner. Otherwise, I don't really know. I mean, in #138 I've observed that changes in an image might affect not only new tag but also older tags - that surprised me a lot in terms of having images for stable/ocata that would have been affected by changes meant for master and later etc. Having versioned docker-compose -f ocata.main.yml -f ocata.metric-pipeline.yml -f ocata.log-pipeline.yml -f ocata.events-pipeline.yml what makes it even more difficult, or actually it may not, is that compose allows to specify those multiple files not only as if we add an extension to core one, but in a manner where we override services' properties. That could mean either...let's see...having a file generated for production case that would simply drop down exposed ports. Having a subdirectories...that sounds nice, it kind of eliminates the issue I lied out above. Though, it still does not eliminate structures like docker-compose -f ocata.main.yml -f ocata.metric-pipeline.yml -f ocata.log-pipeline.yml -f ocata.events-pipeline.yml The difference would be, we just eliminated the ocata prefix, I guess. About the purpose of monasca-notification:
image: monasca/notification:${MON_NOTIFICACATION_VERSION:-"master" we could live without tracking To summarize all that, I think I am kind of attracted to idea to have files called as BTW: Our (Fujitsu) needs are just one side of the coin, flip it and there's a contribution for you that I believe you may take advantage from. I am trying to find some way not just to make it working but also to make it maintainable. The challenge is how to make it so with what we have on the table. BTW2: Another thing, I just remembered of, is that each service you launch, it's description in compose file can be enriched with "mounting" the env file for that particular service. No idea if that helps in the discussion. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Other than figuring out a policy for versions in .env
I think this is good to go.
.env
Outdated
MON_GRAFANA_VERSION=4.0.0-1.1.1 | ||
MON_GRAFANA_INIT_VERSION=1.1.0 | ||
|
||
MON_API_VERSION=master |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It looks like we have some uniquely tagged (semver-ish) images and some that are just master or latest. With this users could end up with an invalid environment if they update from git and don't update any of their master
-tagged images.
I think there's two options here:
- always use unique tags for everything (
master-{date}-{time}
instead of justmaster
)- I tend to prefer this, and we need the unique tags for tagging stable releases (e.g. ocata) and for Helm anyway
- The pr bot can make sure the master branch's
.env
always refers to the latest version of everything- side effect of this is that we periodically re-run CI, for better or for worse
- needs to be manually maintained for a bit though
- document that users should run
docker-compose pull
immediately after running agit pull
- requires few to no updates to either
docker-compose.yml
and.env
into the future - could use
latest
for all semver images too - breaks the ability to check out some arbitrary git revision - can only guarantee a working environment from master or a stable OpenStack release
- rollbacks between git revs are non-trivial
- requires few to no updates to either
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We can always go with the approach that basically allows to override the versions (in any way that it is possible) but does not enforce them in anyway. For example, instead of having
docker-compose.yml:
monasca-persister:
image: monasca/persister:${MON_PERSISTER_VERSION}
.env:
MON_PERSISTER_VERSION=latest
we could have
docker-compose.yml:
monasca-persister:
image: monasca/persister:${MON_PERSISTER_VERSION:-latest}
and just document the possibility of overriding those environmental variables with the reference to official docs.
Still that is somehow not the way that would allow to freeze versions between, for example, each OpenStack release, as there is no file that override those versions. So I think I would go with Option 1. It seems a bit more explicit, which by all means I favor, as there is less room for error because everything is strictly specified. The fact that CI would have to periodically run, I don't think this is too much of a problem. There's an option of having jobs scheduled (as in cron) to run periodically. Based on the docs, the bot could detect that the jobs is periodic and basically do what you just describe above.
Ok, I will rebase the PR and use explicit masters of each image, where it is possible.
docker-compose allows to specify the version of the images used in compose file with environmental variables, as in [1]. This commit is foundation for providing compatibility matrix for different OpenStack releases. [1]: https://docs.docker.com/compose/environment-variables
Commits substitutes rest of images' version with variables names that has been placed inside ```.env``` file. That file will be picked up by docker-compose and will control the latest (master) of the monasca-docker. Added documentation about usage of that new part as well as the possibility of overriding it.
Not all images are now ocata compatible, so there's no point in having tht file.
Instead of relying on implicit images version (i.e. master / latest), use the explicit (strict) version where possible.
@timothyb89 latest commit should feature the changes you requested. It is based on the Option 1 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks good, thanks!
docker-compose allows to specify the version of the images
used in compose file with environmental variables, as in
official docker-compose docs. This commit is foundation for providing compatibility
matrix for different OpenStack releases.
Partially_resolves #135