- refactor: mutualize triggers/functions for upsert into GeoNature database (#44)
With the financial support of Office français de la biodiversité.
**TODO after upgrade: **
Delete triggers and functions on destination database.
DROP TRIGGER IF EXISTS tri_c_upsert_data_to_geonature_with_nomenclature_label ON gn2pg_import.data_json;
DROP TRIGGER IF EXISTS tri_c_upsert_data_to_geonature_with_cd_nomenclature ON gn2pg_import.data_json;
DROP TRIGGER IF EXISTS tri_c_upsert_data_to_geonature_with_metadata ON gn2pg_import.data_json;
DROP TRIGGER IF EXISTS tri_c_delete_data_from_geonature ON gn2pg_import.data_json;
DROP FUNCTION IF EXISTS gn2pg_import.fct_c_get_or_insert_basic_af_from_uuid_name(_uuid UUID, _name TEXT);
DROP FUNCTION IF EXISTS gn2pg_import.fct_c_get_or_insert_basic_dataset_from_uuid_name(_uuid UUID, _name TEXT, _id_af INT);
DROP FUNCTION IF EXISTS gn2pg_import.fct_c_get_or_insert_source(_source TEXT);
DROP FUNCTION IF EXISTS gn2pg_import.fct_c_get_id_nomenclature_from_label(_type TEXT, _label TEXT);
DROP FUNCTION IF EXISTS gn2pg_import.fct_tri_c_upsert_data_to_geonature_with_nomenclature_label();
DROP FUNCTION IF EXISTS gn2pg_import.fct_tri_c_upsert_data_to_geonature_with_cd_nomenclature();
DROP FUNCTION IF EXISTS gn2pg_import.fct_tri_c_delete_data_from_geonature();
DROP FUNCTION IF EXISTS gn2pg_import.fct_c_insert_ds_territories(_id_ds INTEGER, _territories JSONB) ;
DROP FUNCTION IF EXISTS gn2pg_import.fct_c_get_or_create_actors_in_usershub(_actor_role JSONB, _source CHARACTER VARYING) ;
DROP FUNCTION IF EXISTS gn2pg_import.fct_c_insert_dataset_actor(_id_dataset INTEGER, _actor_roles JSONB, _source CHARACTER VARYING);
DROP FUNCTION IF EXISTS gn2pg_import.fct_c_insert_af_actors(_id_af INTEGER, _actor_roles JSONB, _source CHARACTER VARYING) ;
DROP FUNCTION IF EXISTS gn2pg_import.fct_c_get_or_insert_af_from_af_jsondata(_af_data JSONB, _source CHARACTER VARYING);
DROP FUNCTION IF EXISTS gn2pg_import.fct_c_get_or_insert_dataset_from_jsondata(_ds_data JSONB, _id_af INTEGER, _source CHARACTER VARYING);
DROP FUNCTION IF EXISTS gn2pg_import.fct_c_get_or_insert_source(_source TEXT);
DROP FUNCTION IF EXISTS gn2pg_import.fct_tri_c_upsert_data_to_geonature_with_metadata();
DROP FUNCTION IF EXISTS gn2pg_import.fct_tri_c_delete_data_from_geonature();
Apply new to_gnsynthese.sql script.
gn2pg_cli --custom-script to_gnsynthese <myconfigfile>
- New web dashboard using flask-admin (#39)
With the financial support of Office français de la biodiversité.
- Optimization and code factorization (#27 / #29)
- Add automatic tests (#29)
- Improve performances with parallelization of API calls with multithreading (#27 / #30)
- Fix missing custom query strings on updates
- Update base image version in Docker
- Update dependencies
With the financial support of Office français de la biodiversité.
- Add optional query strings on API calls, almost required to order export API using key
orderby
- Update dependencies
- Removal of requests to assign an owner to functions, cause of errors
- Set
gn2pg_import
as default schema name in config template - Fix custom-script command on default to_gnsynthese.sql script due to
%
character - Improve t_roles populate on a GeoNature database using json objects with first and last name
- Fix missing validation status on exports samples
- Fix null value in
gn_synthese.synthese.the_geom_local
caused by null SRID value while getting SRID from first data ingn_synthese.synthese
.
- Fix delete trigger on synthese standard data (
synthese_with_metadata
type missing).
- New SQL scripts for geonature 2 geonature imports which provide triggers to insert data in synthese and populate most of the metadata data (acquisition frameworks, datasets, actors such as organisms and roles, territories, etc.). Source query sample is provided in file geonature_export_sinp_with_metadata.sql
- Fix custom-script error due to % character in sql script (RAISE INFO command).
- Update dependencies
- First official release
- Add forgotten delete trigger on gn_synthese.synthese
- Fix
error_count
type. cf. issue gn2pg_import - error_count donnée en entrée invalide #18
- Fix wrong log function name (previously renamed download_log). cf. issue StorePostgresql object has no attribute log #17.
- new feature: incremental update
- Refactor app name to GN2PG
- fix custom-script that start anytime after --init commands.
- update logos
- Log dowload error in db table
- Add sql custom scripts option to auto populate GeoNature db (provided by default) or anything else.
- Dockerize app
- Cleanup code
- Refactor submodules names
- Improve docs
- Download and store data by offset pages.
- Code cleanup.
First pre-release with full download implemented. This release download all data from API then store in database