Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Deprecated unused/unmaintained components #311

Merged
merged 12 commits into from
Jul 6, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 3 additions & 3 deletions .bumpversion.cfg
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
[bumpversion]
current_version = 1.26.11
current_version = 1.27.0
commit = True
tag = False
tag_name = {new_version}
Expand Down Expand Up @@ -30,11 +30,11 @@ search = {current_version}
replace = {new_version}

[bumpversion:file:RELEASE.txt]
search = {current_version} 2023-07-04T12:57:39Z
search = {current_version} 2023-07-06T16:40:21Z
replace = {new_version} {utcnow:%Y-%m-%dT%H:%M:%SZ}

[bumpversion:part:releaseTime]
values = 2023-07-04T12:57:39Z
values = 2023-07-06T16:40:21Z

[bumpversion:file(version):birdhouse/config/canarie-api/docker_configuration.py.template]
search = 'version': '{current_version}'
Expand Down
8 changes: 8 additions & 0 deletions CHANGES.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,14 @@

[//]: # (list changes here, using '-' for each new entry, remove this when items are added)

[1.27.0](https://github.com/bird-house/birdhouse-deploy/tree/1.27.0) (2023-07-06)
------------------------------------------------------------------------------------------------------------------

- Deprecate unused/unmaintained components

Move unused and unmaintained components to a separate [`deprecated-components/`](birdhouse/deprecated-components)
subdirectory and remove them from the `DEFAULT_CONF_DIRS` list if required.

[1.26.11](https://github.com/bird-house/birdhouse-deploy/tree/1.26.11) (2023-07-04)
------------------------------------------------------------------------------------------------------------------

Expand Down
2 changes: 1 addition & 1 deletion Makefile
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
# Generic variables
override SHELL := bash
override APP_NAME := birdhouse-deploy
override APP_VERSION := 1.26.11
override APP_VERSION := 1.27.0

# utility to remove comments after value of an option variable
override clean_opt = $(shell echo "$(1)" | $(_SED) -r -e "s/[ '$'\t'']+$$//g")
Expand Down
8 changes: 4 additions & 4 deletions README.rst
Original file line number Diff line number Diff line change
Expand Up @@ -14,13 +14,13 @@ for a full-fledged production platform.
* - releases
- | |latest-version| |commits-since|

.. |commits-since| image:: https://img.shields.io/github/commits-since/bird-house/birdhouse-deploy/1.26.11.svg
.. |commits-since| image:: https://img.shields.io/github/commits-since/bird-house/birdhouse-deploy/1.27.0.svg
:alt: Commits since latest release
:target: https://github.com/bird-house/birdhouse-deploy/compare/1.26.11...master
:target: https://github.com/bird-house/birdhouse-deploy/compare/1.27.0...master

.. |latest-version| image:: https://img.shields.io/badge/tag-1.26.11-blue.svg?style=flat
.. |latest-version| image:: https://img.shields.io/badge/tag-1.27.0-blue.svg?style=flat
:alt: Latest Tag
:target: https://github.com/bird-house/birdhouse-deploy/tree/1.26.11
:target: https://github.com/bird-house/birdhouse-deploy/tree/1.27.0

.. |readthedocs| image:: https://readthedocs.org/projects/birdhouse-deploy/badge/?version=latest
:alt: ReadTheDocs Build Status (latest version)
Expand Down
2 changes: 1 addition & 1 deletion RELEASE.txt
Original file line number Diff line number Diff line change
@@ -1 +1 @@
1.26.11 2023-07-04T12:57:39Z
1.27.0 2023-07-06T16:40:21Z
3 changes: 2 additions & 1 deletion birdhouse/components/cowbird/.gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -3,8 +3,9 @@ config/cowbird/config.yml
config/cowbird/cowbird.ini
config/magpie/config.yml
config/proxy/conf.extra-service.d/cowbird.conf
config/proxy/canarie_api_monitoring.py
config/canarie-api/canarie_api_monitoring.py

# Old paths. Keep these so that old config files remain uncommittable after updates.
conf.extra-service.d/cowbird.conf
config/canarie-api/cowbird_config.py
config/proxy/canarie_api_monitoring.py
7 changes: 7 additions & 0 deletions birdhouse/components/cowbird/default.env
Original file line number Diff line number Diff line change
Expand Up @@ -36,3 +36,10 @@ export COWBIRD_MONGODB_PORT=27017

# Subdirectory containing the user workspaces used by Cowbird
export USER_WORKSPACES="user_workspaces"

# this dependency is only required if COWBIRD_MONGODB_SERVICE is the one provided in config/mongodb.
# (include this by default to support backwards compatibility for now)
COMPONENT_DEPENDENCIES="
$COMPONENT_DEPENDENCIES
./config/mongodb
"
8 changes: 8 additions & 0 deletions birdhouse/config/.gitignore
Original file line number Diff line number Diff line change
@@ -1,2 +1,10 @@
# Old paths. Keep these so that old config files remain uncommittable after updates.
postgres-magpie/
catalog/
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for remembering this to not break autodeploy !

frontend/
malleefowl/
ncops/
ncwms2/
project-api/
solr/
phoenix/
Original file line number Diff line number Diff line change
Expand Up @@ -109,8 +109,8 @@ SERVICES = {
# NOTE:
# Below version and release time auto-managed by 'make VERSION=x.y.z bump'.
# Do NOT modify it manually. See 'Tagging policy' in 'birdhouse/README.rst'.
'version': '1.26.11',
'releaseTime': '2023-07-04T12:57:39Z',
'version': '1.27.0',
'releaseTime': '2023-07-06T16:40:21Z',
'institution': 'Ouranos',
'researchSubject': 'Climatology',
'supportEmail': '${SUPPORT_EMAIL}',
Expand Down Expand Up @@ -142,8 +142,8 @@ PLATFORMS = {
# NOTE:
# Below version and release time auto-managed by 'make VERSION=x.y.z bump'.
# Do NOT modify it manually. See 'Tagging policy' in 'birdhouse/README.rst'.
'version': '1.26.11',
'releaseTime': '2023-07-04T12:57:39Z',
'version': '1.27.0',
'releaseTime': '2023-07-06T16:40:21Z',
'institution': 'Ouranos',
'researchSubject': 'Climatology',
'supportEmail': '${SUPPORT_EMAIL}',
Expand Down

This file was deleted.

This file was deleted.

This file was deleted.

This file was deleted.

This file was deleted.

This file was deleted.

This file was deleted.

This file was deleted.

This file was deleted.

This file was deleted.

This file was deleted.

This file was deleted.

1 change: 1 addition & 0 deletions birdhouse/config/thredds/.gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@ threddsConfig.xml
config/proxy/conf.extra-service.d/thredds.conf
config/canarie-api/canarie_api_monitoring.py
service-config.json
config/magpie/providers.cfg

# Old paths. Keep these so that old config files remain uncommittable after updates.
config/proxy/canarie_api_monitoring.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
version: "3.4"
services:
magpie:
volumes:
- ./config/thredds/config/magpie/providers.cfg:${MAGPIE_PROVIDERS_CONFIG_PATH}/thredds-providers.cfg:ro
35 changes: 35 additions & 0 deletions birdhouse/config/thredds/config/magpie/providers.cfg.template
Original file line number Diff line number Diff line change
@@ -0,0 +1,35 @@
providers:
thredds:
url: http://thredds:8080/twitcher/ows/proxy/thredds
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same as per other comment.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is not included in #331 since this is being moved from the catalog component.

If you'd like I can make this use the external port 8083 and then change it again when/if #331 gets pulled in but it seems like we might as well do it right the first time (in this PR) since this is new here.

title: Thredds
public: true
c4i: false
type: thredds
sync_type: thredds
# below is a custom config to indicate how magpie should convert thredds path elements into resources/permissions
# see: https://pavics-magpie.readthedocs.io/en/latest/services.html#servicethredds
configuration:
skip_prefix: "thredds" # prefix to ignore, below prefixes will be matched against whatever comes after in path
file_patterns:
# note: make sure to employ quotes and double escapes to avoid parsing YAML error
- ".+\\.ncml" # match longest extension first to avoid tuncating it by match of sorter '.nc'
- ".+\\.nc"
metadata_type:
prefixes:
- null # note: special YAML value evaluated as `no-prefix`, use quotes if literal value is needed
- "\\w+\\.gif" # threddsIcon, folder icon, etc.
- "\\w+\\.ico" # favicon
- "\\w+\\.txt" # licence
- "\\w+\\.css" # tds.css
- "catalog\\.\\w+" # note: special case for `THREDDS` top-level directory (root) accessed for `BROWSE`
- catalog
- ncml
- uddc
- iso
data_type:
prefixes:
- fileServer
- dodsC
- wcs
- wms
- ncss
6 changes: 0 additions & 6 deletions birdhouse/default.env
Original file line number Diff line number Diff line change
Expand Up @@ -39,7 +39,6 @@ or a custom web interface.
export DEFAULT_CONF_DIRS='
./config/proxy
./config/canarie-api
./config/phoenix
./config/geoserver
./config/flyingpigeon
./config/finch
Expand All @@ -50,9 +49,4 @@ export DEFAULT_CONF_DIRS='
./config/magpie
./config/twitcher
./config/jupyterhub
./config/frontend
./config/project-api
./config/catalog
./config/malleefowl
./config/solr
'
29 changes: 29 additions & 0 deletions birdhouse/deprecated-components/README.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
Deprecated Components
#####################

.. contents::

All components in this directory are not actively maintained.
mishaschwartz marked this conversation as resolved.
Show resolved Hide resolved

If you wish to include these components in the deployed stack, additional work
may be required to make them compatible with the current stack.

Components in this directory may be removed from this repository at any time.

This directory also contains additional configurations for these deprecated components in:

- `all-public-access-deprecated/`
- `canarie-api-full-monitoring-deprecated/`
- `wps-healthchecks-deprecated/`
Comment on lines +13 to +17
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thinking about this again... Is there actually any reason to move them under a separate configuration?

If I activated for example:

EXTRA_CONF_DIRS="
  ./components/weaver
  ./deprecated-components/malleefowl
  ./optional-components/all-public-access-deprecated
  ./optional-components/canarie-api-full-monitoring
  ./optional-components/wps-healthchecks
"

Wouldn't the following be loaded (not necessarily exhaustive or in the right order below)?

./components/weaver/*
./deprecated-components/malleefowl
./optional-components/all-public-access-deprecated/mallefowl/*
./optional-components/all-public-access-deprecated/weaver/*
./optional-components/canarie-api-full-monitoring/mallefowl/*
./optional-components/canarie-api-full-monitoring/weaver/*
./optional-components/wps-healthchecks/mallefowl/*
./optional-components/wps-healthchecks/weaver/*

In order words, whether the main services are under config, components or deprecated-components, the optional-components to use would load whichever service name that it matches nested under its directory.

I find this would be easier to maintain because a service moving between config, components or deprecated-components states would not require updating the optional-components, nor require loading another (optional+deprecated)-component variant equivalent to the original one.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wouldn't the following be loaded (not necessarily exhaustive or in the right order below)?

At the moment yes but that is a mistake. I just forgot to delete the catalog and malleefowl directories under optional-components/wps-healthchecks/

I've remove them now.

I find this would be easier to maintain because a service moving between config, components or deprecated-components states would not require updating the optional-components, nor require loading another (optional+deprecated)-component variant equivalent to the original one.

I think that you need to think about whether you actually want to deprecate these components or not. In my opinion, the point of moving them into a separate folder is to clearly mark that they are deprecated and no longer maintained. Developers then can focus on maintaining the other components without worrying about breaking components that we don't care about anymore.

The only reason I can think of to keep supporting code in the optional-components folder is if you think that these components will be "un-deprecated" at some point.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If someone needs to work on, for example, optional-components/wps-healthchecks to add a new WPS service, this doesn't require them to update all nested components. The optional-components/wps-healthchecks/malleefowl for example would be left untouched. I don't think leaving them under optional-components will cause any breaking issue, nor would affect maintenance burden.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Up to you... I moved them back


These contain the settings to extend the deprecated components that have been moved from the corresponding
directories under `birdhouse/optional-components`.

To enable these additional configurations; add them to the `EXTRA_CONF_DIRS` variable (in `env.local`)
as you would to enable any component. For example, to enable the deprecated malleefowl component as well as the
wps-healthchecks for malleefowl. The `EXTRA_CONF_DIRS` variable should contain:

.. code-block:: shell

./deprecated-components/malleefowl
./wps-healthchecks-deprecated
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
version: "3.4"
services:
proxy:
volumes:
- ./deprecated-components/catalog/config/canarie-api/canarie_api_monitoring.py:${CANARIE_MONITORING_EXTRA_CONF_DIR}/catalog_canarie_api_monitoring.py:ro
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
version: "3.4"
services:
magpie:
volumes:
- ./deprecated-components/catalog/config/magpie/providers.cfg:${MAGPIE_PROVIDERS_CONFIG_PATH}/catalog-providers.cfg:ro
- ./deprecated-components/catalog/config/magpie/permissions.cfg:${MAGPIE_PERMISSIONS_CONFIG_PATH}/catalog-permissions.cfg:ro
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ services:
ports:
- "8086:80"
volumes:
- ./config/catalog/catalog.cfg:/home/catalog.cfg
- ./deprecated-components/catalog/catalog.cfg:/home/catalog.cfg
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice ! These deprecated should still work !

- wps_outputs:/var/www/html/wps_results
depends_on:
- postgres
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
version: "3.4"
services:
proxy:
volumes:
- ./deprecated-components/frontend/config/canarie-api/canarie_api_monitoring.py:${CANARIE_MONITORING_EXTRA_CONF_DIR}/frontend_canarie_api_monitoring.py:ro
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
version: "3.4"
services:
proxy:
volumes:
- ./deprecated-components/frontend/config/proxy/conf.extra-service.d:/etc/nginx/conf.extra-service.d/frontend:ro
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,6 @@ services:
ports:
- "3000:3000"
env_file:
- ./config/frontend/frontend.env
- ./deprecated-components/frontend/frontend.env
restart: always
logging: *default-logging
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
version: "3.4"
services:
proxy:
volumes:
- ./deprecated-components/malleefowl/config/canarie-api/canarie_api_monitoring.py:${CANARIE_MONITORING_EXTRA_CONF_DIR}/malleefowl_canarie_api_monitoring.py:ro
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
version: "3.4"
services:
magpie:
volumes:
- ./deprecated-components/malleefowl/config/magpie/providers.cfg:${MAGPIE_PROVIDERS_CONFIG_PATH}/malleefowl-providers.cfg:ro
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@ services:
- "48091:9001"
volumes:
- ${DATA_PERSIST_ROOT}/datasets:/pavics-data
- ./config/malleefowl/custom.cfg:/opt/birdhouse/src/malleefowl/custom.cfg
- ./deprecated-components/malleefowl/custom.cfg:/opt/birdhouse/src/malleefowl/custom.cfg
depends_on:
- postgres
restart: always
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
version: "3.4"
services:
magpie:
volumes:
- ./deprecated-components/ncwms2/config/magpie/providers.cfg:${MAGPIE_PROVIDERS_CONFIG_PATH}/ncwms2-providers.cfg:ro
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
version: "3.4"
services:
proxy:
volumes:
- ./deprecated-components/ncwms2/config/proxy/conf.extra-service.d:/etc/nginx/conf.extra-service.d/ncwms2:ro
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ services:
- "48080:9001"
volumes:
- ${DATA_PERSIST_ROOT}/datasets:/pavics-data
- ./config/ncwms2/custom.cfg:/opt/birdhouse/custom.cfg
- ./config/ncwms2/server.xml:/opt/birdhouse/eggs/birdhousebuilder.recipe.tomcat-0.2.9-py2.7.egg/birdhousebuilder/recipe/tomcat/server.xml
- ./deprecated-components/ncwms2/custom.cfg:/opt/birdhouse/custom.cfg
- ./deprecated-components/ncwms2/server.xml:/opt/birdhouse/eggs/birdhousebuilder.recipe.tomcat-0.2.9-py2.7.egg/birdhousebuilder/recipe/tomcat/server.xml
restart: always
logging: *default-logging
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
version: "3.4"
services:
proxy:
volumes:
- ./deprecated-components/phoenix/config/canarie-api/canarie_api_monitoring.py:${CANARIE_MONITORING_EXTRA_CONF_DIR}/phoenix_canarie_api_monitoring.py:ro
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@ services:
- "38443:38443"
- "9001:9001"
volumes:
- ./config/phoenix/custom.cfg:/opt/birdhouse/src/phoenix/custom.cfg
- ./deprecated-components/phoenix/custom.cfg:/opt/birdhouse/src/phoenix/custom.cfg
- ${SSL_CERTIFICATE}:/opt/birdhouse/etc/nginx/cert.pem
links:
- mongodb
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
version: "3.4"
services:
proxy:
volumes:
- ./deprecated-components/project-api/config/canarie-api/canarie_api_monitoring.py:${CANARIE_MONITORING_EXTRA_CONF_DIR}/project-api_canarie_api_monitoring.py:ro
Loading