Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

postgresql versions moved to common facts #600

Merged
merged 2 commits into from
Aug 10, 2022
Merged

postgresql versions moved to common facts #600

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
f6ed467
postgresql versions moved to common facts
vjrj Jul 15, 2022
2840ec6
Only install postgis if need it
vjrj Aug 5, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion ansible/data_quality_filter_service.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@
- common
- java
- {role: db-backup, db: postgres, db_name: "{{ dq_db_name }}", db_user: "{{ dq_db_user }}", db_password: "{{ dq_db_password }}" }
- {role: postgresql, pg_version: "{{ data_quality_pg_version | default('13') }}"}
- postgresql
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

instead of removing, why not just define data_quality_pg_version in the relevant inventory?

- {role: pg_instance, extensions: [], db_name: "{{ dq_db_name }}", db_user: "{{ dq_db_user }}", db_password: "{{ dq_db_password }}" }
- tomcat
- webserver
Expand Down
2 changes: 1 addition & 1 deletion ansible/doi-service-standalone.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@
- java
- postfix
- {role: db-backup, db: postgres, db_name: "{{ doi_db_name }}", db_user: "{{ doi_db_user }}", db_password: "{{ doi_db_password }}" }
- {role: postgresql, pg_version: "9.6"}
- postgresql
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

instead of removing the version, why not just add doi_service_pg_version with a default and override it in the relevant inventory?

- {role: pg_instance, extensions: ["citext", "pgcrypto"], db_name: "{{ doi_db_name }}", db_user: "{{ doi_db_user }}", db_password: "{{ doi_db_password }}" }
- {role: ansible-elasticsearch, es_templates: false, es_instance_name: 'doi-elasticsearch', es_data_dirs: ['/data/elasticsearch'], tags: ['elasticsearch']}
- webserver
Expand Down
4 changes: 3 additions & 1 deletion ansible/image-service.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,10 +1,12 @@
- hosts: image-service
vars:
use_postgis: true
roles:
- java
- common
- { role: ansible-elasticsearch, es_templates: false, es_instance_name: 'images', tags: ['elasticsearch']}
- { role: db-backup, db: postgres, db_name: "images", db_user: "{{ image_service_db_username }}", db_password: "{{ image_service_db_password }}" }
- { role: postgresql, pg_version: "{{ image_service_postgresql_version | default('10') }}", postgis_version: "{{ image_service_postgis_version | default('2.4') }}" }
- postgresql
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

instead of removing, why not just define image_service_postgresql_version and image_service_postgis_version in the relevant inventory?

- { role: pg_instance, extensions: ["citext", "pgcrypto"], db_name: "{{ image_service_db_name }}", db_user: "{{ image_service_db_username }}", db_password: "{{ image_service_db_password }}", db_role_attr_flags: "NOSUPERUSER" }
- webserver
- i18n
Expand Down
2 changes: 2 additions & 0 deletions ansible/phylolink.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,6 @@
- hosts: phylolink
vars:
use_postgis: true
roles:
- phylolink-dependencies
- common
Expand Down
9 changes: 9 additions & 0 deletions ansible/roles/common/tasks/setfacts.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -87,6 +87,9 @@
python_software_properties: "python3-software-properties"
tomcat_dir: "/var/lib/{{ tomcat }}"
tomcat_webapps: "/var/lib/{{ tomcat }}/webapps/"
pg_version: '9.6'
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some apps will have minimum postgres versions, eg they'll be using jsonb or something introduced in pg10+. Why force 9.6 here?

postgis_version: '2.4'
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This seems wrong - the postgres role is not supposed to install postgis unless the postgis version is defined. By defining it here, won't postgis always be installed?

postgis_scripts_version: '2.4'
# openjdk-8 will be removed at some point from Ubuntu-18.04, so we will need to switch to Zulu-8 or similar at that point
# openjdk-lts on Ubuntu-18.04 is JDK-11, so will not be directly useful by our applications at this point due to lack of testing
# use_openjdk8: true
Expand Down Expand Up @@ -124,6 +127,9 @@
python_software_properties: "python3-software-properties"
tomcat_dir: "/var/lib/{{ tomcat }}"
tomcat_webapps: "/var/lib/{{ tomcat }}/webapps/"
pg_version: '10'
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ditto above

postgis_version: '2.5'
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ditto above

postgis_scripts_version: '2.5'
# openjdk-8 will be removed at some point from Ubuntu-18.04, so we will need to switch to Zulu-8 or similar at that point
# use_openjdk8: true
# use_zulu8: False
Expand Down Expand Up @@ -161,6 +167,9 @@
tomcat_dir: "/var/lib/{{ tomcat }}"
tomcat_webapps: "/var/lib/{{ tomcat }}/webapps/"
psycopg2_package: "python3-psycopg2"
pg_version: '14'
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ditto above

postgis_version: '3'
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ditto above

postgis_scripts_version: '3.2'
when: ansible_os_family == "Debian" and ansible_distribution_version == "22.04"
tags:
- setfacts
Expand Down
1 change: 1 addition & 0 deletions ansible/roles/postgresql/defaults/main.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
install_postgis: "{{ use_postgis | default(False) }}"
6 changes: 3 additions & 3 deletions ansible/roles/postgresql/tasks/debian.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@
- name: Set postgis package names
set_fact:
postgresql_packages: "{{postgresql_packages}} + [ 'postgresql-{{pg_version}}-postgis-{{postgis_version}}', 'postgresql-{{pg_version}}-postgis-{{postgis_version}}-scripts', 'gdal-bin' ]"
when: postgis_version is defined
when: postgis_version is defined and install_postgis | bool == True
tags: postgresql

- name: Install Postgres
Expand Down Expand Up @@ -55,7 +55,7 @@
# Not pulling in recommended packages as that would pull in postgres-10
- name: Install separate postgis package to get shp2pgsql binary
apt: pkg="postgis" state="present" update_cache="yes" install_recommends="no"
when: postgis_version is defined
when: postgis_version is defined and install_postgis | bool == True
tags:
- postgresql
register: postgresql_installed
Expand All @@ -65,7 +65,7 @@
# PostGIS 2.2+ only needs CREATE EXTENSION postgis; CREATE EXTENSION postgis_topology; etc on the target DB
- debug:
msg: "PostGIS-enabled databases also need a corresponding pg_instance role with postgis in the extensions list"
when: postgis_version is defined
when: postgis_version is defined and install_postgis | bool == True

- name: start postgresql and configure to startup automatically
service:
Expand Down
21 changes: 14 additions & 7 deletions ansible/roles/postgresql/tasks/postgis.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,8 +1,15 @@
- name: "Show postgis version"
debug:
msg: "postgis version: {{ postgis_version }} and installing postgis: {{ install_postgis }}"
tags:
- postgresql
- postgis

- name: install setfacl support
become: yes
apt:
pkg: acl
when: postgis_version
when: postgis_version and install_postgis | bool == True
tags:
- postgresql
- postgis
Expand All @@ -24,7 +31,7 @@
postgresql_db:
db: postgis_template
state: present
when: postgis_version
when: postgis_version and install_postgis | bool == True
become: yes
become_user: postgres
tags:
Expand All @@ -33,7 +40,7 @@

- name: make postgis_template a template
action: command psql -d postgis_template -c "UPDATE pg_database SET datistemplate=true WHERE datname='postgis_template';"
when: postgis_version
when: postgis_version and install_postgis | bool == True
become: yes
become_user: postgres
tags:
Expand All @@ -42,13 +49,13 @@

- name: run the postgis SQL scripts
action: command psql -d postgis_template -f {{ item }}
when: postgis_version
when: postgis_version and install_postgis | bool == True
become: yes
become_user: postgres
with_items:
- "/usr/share/postgresql/{{pg_version}}/contrib/postgis-{{postgis_version}}/postgis.sql"
- "/usr/share/postgresql/{{pg_version}}/contrib/postgis-{{postgis_version}}/spatial_ref_sys.sql"
- "/usr/share/postgresql/{{pg_version}}/contrib/postgis-{{postgis_version}}/postgis_comments.sql"
- "/usr/share/postgresql/{{pg_version}}/contrib/postgis-{{postgis_scripts_version}}/postgis.sql"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we remove this action and just add postgis to the extensions list in the corresponding pg_instance calls, since nothing is using postgis < 2.2 and this is only necessary for those postgis versions AFAIK?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok. Pending

- "/usr/share/postgresql/{{pg_version}}/contrib/postgis-{{postgis_scripts_version}}/spatial_ref_sys.sql"
- "/usr/share/postgresql/{{pg_version}}/contrib/postgis-{{postgis_scripts_version}}/postgis_comments.sql"
tags:
- postgresql
- postgis
4 changes: 3 additions & 1 deletion ansible/spatial.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,11 +1,13 @@
- name: spatial
hosts: spatial
vars:
use_postgis: true
roles:
- common
- java
- tomcat
- webserver
- {role: postgresql, pg_service: 'postgresql-9.6', pg_version: '9.6', postgis_version: '2.4'}
- postgresql
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

instead of removing the version, why not add spatial_pg_version and sptaial_postgis_version and then override them in the relevant inventory?

- {role: pg_instance, extensions: ['postgis', 'postgis_topology', 'uuid-ossp'], db_name: "{{geonetwork_database}}", db_user: "{{geonetwork_db_username}}", db_password: "{{geonetwork_db_password }}" }
- {role: pg_instance, extensions: ['postgis', 'postgis_topology', 'uuid-ossp'], db_name: "{{layers_db_name}}", db_user: "{{layers_db_username}}", db_password: "{{layers_db_password}}" }
- layers-db
Expand Down