-
Notifications
You must be signed in to change notification settings - Fork 443
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Support Galera Replication #28
Comments
Would Galera cover that for you (#24) or are you looking at the autoconfiguration that is in their script (https://github.com/bitnami/bitnami-docker-mariadb/blob/master/rootfs/bitnami-utils-custom.sh)? |
I am on the same path mariadb has this build it from 10.1 and after |
If anyone is interested - this is not very straightforward, but doable. BTW, I'm not sure that that PR with Galera support is really as good as it could be. Also I have such feeling, that official image should not provide Galera setup, it is tricky and will not be flexible enough eventually. Current image is good enough and contains most of necessary basic blocks to build Galera image yourself while nicely reusing what is done here already. |
thanks I will check it. I will not advise you to go for the GlusterFS , especialy for web apps. I haven't tested it myself , but the feedback is that is quite slow for when accessing many small files. I have setup 2x lsyncd container and so far it syncs really well. It is kind of a rsync daemon , watches for changes and it fires rsync The documentation sais not fit for 2 way sync , but in my setup it works very , very well. Here is the repo, It is not universal but you don't have time right now to improve it. I start is with the same compose file on both hosts lsyncd: it needs 2 env variables on host 1 on host 2 |
If you're going to support Galera, please be sure to include |
The current 10.x builds already have the Galera plugin. Doesn't that work? I have been trying to debug that for quite a while now and the second node after the bootstrap node crashes saying |
I have been using the MariaDB 10.1 with Galera for a long time (3+ months) and it works perfect..
I run it on a CoreOS cluster with Skydns, and the server register itself with Skydns/etcd so it is available on "mysql.skydns.local".. |
@klausenbusk Would you mind quickly reviewing the configs present at the SO link I have provided? http://stackoverflow.com/questions/39744949/unable-to-create-mariadb-galera-cluster |
First you should use 10.1, secondly it doesn't seems like you expose any ports?, thirdly you need to set --wsrep-node-address.. :) |
@klausenbusk I am using inter-container networking so I don't think port exposure is needed. I am trying to run it on my local machine, I can worry about multi-host communication later. Also |
@activatedgeek Oh yes, I think you correct :) |
@klausenbusk Have a look here: http://pastebin.com/3exPpqvc. If you have a look at lines 215-217 you can see that the SST was completed successfully, but the node crashed due to failed init. Also line 220 shows |
The issues is that the image get stuck in the initializing phase (which is running with --skip-networking (explain port 0)). So try something like:
|
@klausenbusk So essentially, you mean that start all nodes such that the folder |
Correct expect for the bootstrap node, also you should consider using xtrabackup for SST. |
@klausenbusk Perfect. This is working great now. I'll create a wrapper MariaDB image which takes an extra environment flag to create the dir for non-bootstrap nodes. Thank you so much for your inputs! |
I believe tanji had a working setup with this compose file: #57 (comment). |
Not sure if this relates to this issue, but I get this warning on new database initialization:
|
@strarsis Unrelated, please open a different issue for that |
I think supporting replication out of the box is probably a bit too ambitious for this image (as noted by other folks above) -- there are too many edge cases and environmental configuration for us to do that really well in such a way that would satisfy not only existing replication users but also cover the use cases of new users. |
Severalnines have an example for docker-swarm in severalnines/galera-docker-mariadb. They also have example for kubernetes there, but I had problems with the etcd (there is only incubator chart for it and it seems broken), so I rewrote it to use labels in kubernetes instead, and used this image. Notable differences are that they are still using xtracbackup-v2, while this image (only) has mariabackup, and different path to the galera_ssm library. Version 10.2 also changes the state variables, so the healthcheck scripts need to be rewritten as severalnines still have 10.1. I used this image unmodified, just injecting a script wrapping the default entrypoint from the kubernetes manifest. It is kubernetes-specific, so appropriate place to host it would be the helm/charts. I had one issue with the docker-entrypoint.sh though: I prefixed it with a bit that derives the appropriate |
Most discussion here seems galera related. MDEV-25667 recently got raised on getting some better support for doing galera in the server that will with hopefully minimal entrypoint changes provide the necessary functionality. galera-arbitrator should be its own image rather than bloating this one. |
I have some little success in this area. My experiences:
Working results can be found here: https://gitlab.com/tymonx/docker-mariadb. I have a simple Docker Compose example with automatic bootstrapping and automatic cluster nodes join. This should work out-of-box without requiring a third party (especially an additional cluster controller/manager). There are some TODOs that can be done like:
|
It will definitely help.
This can be solved. By detecting this and creating a script wrapper that will execute both (or more) scripts, the current one (always forked) and custom one provided by user. It is similar to observer pattern from software when you register N custom callbacks and fire them in loop for every event/notification. This case also happens in classic signal (C?) or trap (POSIX scripts) mechanism in application and scripts. And using the observer pattern helps to resolve that. |
After some thinking, I have a nice proposition for the |
yes, single command (implementation of execution) |
I think you're over engineering it (MDEV-25742). If a user wants their own notification script as well as what you what you develop, they can wrap it in a script themselves. If really needed its a rather simple shell script to fan out an reap, lets not burden the server the the added complexity. |
If you think this feature will not help or make something easier I can close it. At beginning the idea sounds good :) I have prepared a simple test. When I was using official image My tests can be found here: https://gitlab.com/tymonx/docker-mariadb/-/tree/dev-mariadb-docker-initial-auto-join-failure If you want look into CI logs:
|
I think if you finish to implement this feature (MDEV-25667) and add support for the |
Good news! After updating the I have added only the auto bootstrap feature. My working replication support https://gitlab.com/tymonx/docker-mariadb:
|
quick note: 6f5d272 is the cause of the timezone initialization failures in your CI. A 10.6.1 release is imitate. You want work around it with Chat available on https://mariadb.zulipchat.com |
Thanks! I have recently tested with Docker Compose and Docker Swarm. Current mysqld-entrypoint implementation. Docker Compose scaling:
|
Add scaling using external
|
ok. Looking forward to a PR. Notes based on current entrypoint:
|
Sure :) I will fork this project and prepare a proper PR including your suggestions.
It is related with the SC2068 warning. The |
This patch add support for Galera replication. Features: - it detects if Galera replication was enabled using `mysql` configuration files or provided `mysqld` command line arguments - on default it enables cluster auto bootstrap feature - on default the first cluster node is used for cluster auto bootstrapping based on the wsrep_cluster_address parameter from `mysql` configuration files, `mysqld` command line arguments or by setting the `WSREP_CLUSTER_ADDRESS` environment variable - cluster auto bootstrap feature can be disabled by setting the `WSREP_SKIP_AUTO_BOOTSTRAP` environment variable - use the `WSREP_AUTO_BOOTSTRAP_ADDRESS` environment variable to explicitly choice other node for cluster bootstrapping - cluster node hostnames or IP addresses must be valid to enable cluster auto bootstrapping How to use it. 1. Prepare `mysql` configuration file `galera.cnf`: ```plaintext [galera] wsrep_on = ON wsrep_sst_method = rsync wsrep_provider = /usr/lib/libgalera_smm.so bind-address = 0.0.0.0 binlog_format = row default_storage_engine = InnoDB innodb_doublewrite = 1 innodb_autoinc_lock_mode = 2 innodb_flush_log_at_trx_commit = 2 ``` 2. Make it read-only: ```plaintext chmod 444 galera.cnf ``` 3. Prepare Docker Compose file `docker-compose.yml`: ```yaml services: node: image: mariadb restart: always security_opt: - label=disable environment: WSREP_CLUSTER_ADDRESS: "${WSREP_CLUSTER_ADDRESS:-}" MYSQL_ROOT_PASSWORD: example volumes: - ./galera.cnf:/etc/mysql/conf.d/10-galera.cnf:ro command: - --wsrep-cluster-address=gcomm://db_node_1,db_node_2,db_node_3 deploy: replicas: 3 ``` 4. Start Docker Compose: ```plaintext docker-compose --project-name db up ``` To start N MariaDB instances using environment variable: ```plaintext WSREP_CLUSTER_ADDRESS="gcomm://db_node_1,db_node_2,db_node_3,db_node_4,db_node_5" docker-compose --project-name db up --scale node="$(echo "${WSREP_CLUSTER_ADDRESS}" | tr ',' ' ' | wc -w)" ``` To start N MariaDB instances using `mysql` configuration file: ```plaintext docker-compose --project-name db up --scale node="$(grep -i wsrep_cluster_address <name>.cnf | tr -d ' ' | tr ',' ' ' | wc -w)" ```
@grooverdan done. Ready for review :) |
I have added two useful environment variables I can reverse the logic and disable the auto bootstrap feature on default. Then I can replace the |
This patch add support for Galera replication. Features: - It detects if Galera replication was enabled wsrep_on=ON - By default it enables cluster auto bootstrap feature - By default the first cluster node is used for cluster auto bootstrapping based on the wsrep_cluster_address parameter or by setting the `WSREP_CLUSTER_ADDRESS` environment variable - cluster auto bootstrap feature can be disabled by setting the `WSREP_SKIP_AUTO_BOOTSTRAP` environment variable - use the `WSREP_AUTO_BOOTSTRAP_ADDRESS` environment variable to explicitly choice other node for cluster bootstrapping - cluster node hostnames or IP addresses must be valid to enable cluster auto bootstrapping How to use it. 1. Prepare MariaDB configuration file `galera.cnf`: ```plaintext [galera] wsrep_on = ON wsrep_sst_method = mariabackup wsrep_provider = /usr/lib/libgalera_smm.so binlog_format = row default_storage_engine = InnoDB innodb_doublewrite = 1 innodb_autoinc_lock_mode = 2 ``` 2. Make it read-only: ```plaintext chmod 444 galera.cnf ``` 3. Prepare Docker Compose file `docker-compose.yml`: ```yaml services: node: image: mariadb restart: always security_opt: - label=disable environment: WSREP_CLUSTER_ADDRESS: "${WSREP_CLUSTER_ADDRESS:-}" MARIADB_ROOT_PASSWORD: example volumes: - ./galera.cnf:/etc/mysql/conf.d/10-galera.cnf:ro command: - --wsrep-cluster-address=gcomm://db_node_1,db_node_2,db_node_3 deploy: replicas: 3 ``` 4. Start Docker Compose: ```plaintext docker-compose --project-name db up ``` To start N MariaDB instances using environment variable: ```plaintext WSREP_CLUSTER_ADDRESS="gcomm://db_node_1,db_node_2,db_node_3,db_node_4,db_node_5" docker-compose --project-name db up --scale node="$(echo "${WSREP_CLUSTER_ADDRESS}" | tr ',' ' ' | wc -w)" ``` To start N MariaDB instances using MariaDB configuration file: ```plaintext docker-compose --project-name db up --scale node="$(grep -i wsrep_cluster_address <name>.cnf | tr -d ' ' | tr ',' ' ' | wc -w)" ``` Closes: MariaDB#28
This patch add support for Galera replication. Features: - It detects if Galera replication was enabled wsrep_on=ON - By default it enables cluster auto bootstrap feature - By default the first cluster node is used for cluster auto bootstrapping based on the wsrep_cluster_address parameter or by setting the `WSREP_CLUSTER_ADDRESS` environment variable - cluster auto bootstrap feature can be disabled by setting the `WSREP_SKIP_AUTO_BOOTSTRAP` environment variable - use the `WSREP_AUTO_BOOTSTRAP_ADDRESS` environment variable to explicitly choice other node for cluster bootstrapping - cluster node hostnames or IP addresses must be valid to enable cluster auto bootstrapping How to use it. 1. Prepare MariaDB configuration file `galera.cnf`: ```plaintext [galera] wsrep_on = ON wsrep_sst_method = mariabackup wsrep_provider = /usr/lib/libgalera_smm.so binlog_format = row default_storage_engine = InnoDB innodb_doublewrite = 1 innodb_autoinc_lock_mode = 2 ``` 2. Make it read-only: ```plaintext chmod 444 galera.cnf ``` 3. Prepare Docker Compose file `docker-compose.yml`: ```yaml services: node: image: mariadb restart: always security_opt: - label=disable environment: WSREP_CLUSTER_ADDRESS: "${WSREP_CLUSTER_ADDRESS:-}" MARIADB_ROOT_PASSWORD: example volumes: - ./galera.cnf:/etc/mysql/conf.d/10-galera.cnf:ro command: - --wsrep-cluster-address=gcomm://db_node_1,db_node_2,db_node_3 deploy: replicas: 3 ``` 4. Start Docker Compose: ```plaintext docker-compose --project-name db up ``` To start N MariaDB instances using environment variable: ```plaintext WSREP_CLUSTER_ADDRESS="gcomm://db_node_1,db_node_2,db_node_3,db_node_4,db_node_5" docker-compose --project-name db up --scale node="$(echo "${WSREP_CLUSTER_ADDRESS}" | tr ',' ' ' | wc -w)" ``` To start N MariaDB instances using MariaDB configuration file: ```plaintext docker-compose --project-name db up --scale node="$(grep -i wsrep_cluster_address <name>.cnf | tr -d ' ' | tr ',' ' ' | wc -w)" ``` Closes: MariaDB#28
The existing image is supporting Galera. The Galeria library is included in the image. I have made it works for my NextCloud. I even have the maxscale running in front of these cluster nodes. I am using k8s (on RPI4s :-) ) so I will just highlight those changes needed and I believe you can easily replicates to docker and so on. You need to run the container with the following command. This is the syntax for YAML, should be very similar to docker compose. Use the following to start the master node. Launch your slave nodes using standard approach as what you always do. Best to wait for your master node to be ready then only launching the slave nodes.
The following is the config required in
|
@chengkuangan But then you have to edit the config every time you want to add a new node, no? If you stop the cluster, how do you restart it again? For me it will always complain, since the cluster already have been initialized and Thanks for taking the time to share this with us :) |
@skjnldsv From what I read if we stop the cluster completely, it considered terminated, we will need to bootstrap the cluster again. Which is mentioned in the doc. It will be a manual step. Thanks for pointing of removing the To be frank, I am not an expert here... This is my first time setting up MariaDB cluster and maxscale. and it is RPI4 playground ... so no guaranteed I am 100% correct. I am just sharing what I have learned and subject to mistake. :-) |
I forgot to mention. I have to initiate all the PODs first time without galera first. When the pod is ready, I will change it to enable galena and restart it. Reason of doing so is if galera is enabled the first time, the instance will failed with error complaining mysql tables and other files are not available. I guess this maybe because the |
oh another thing, I have not looked hard enough maybe ... so far I can't find the Any idea? |
There's some notes started on grastate.dat and documentation references on MDEV-25855. Insights and corrections welcome. |
@grooverdan To verify ... |
You can generate it with |
I'm still learning the mechanics of the mechanisms available |
I recently have a crash. I follow the documented approach and be able to bring the cluster back without any problem. Thanks for all the tips and guides here. This is what I do:
|
To use this docker image in production, it would be very nice to be able to run it with replication, to maintain some redundancy.
The bitnami/mariadb image already does this, but they don't have the 10.x versions.
The text was updated successfully, but these errors were encountered: