Dev suivi eolien rebased on develop #238

andriacap · 2023-07-24T07:36:02Z

Cette branche a été rebase par rapport à la branche branche develop de https://github.com/PnX-SI/gn_module_monitoring . Et a été installé sur le GN 2.13.0-beta1 (branche develop dernier commit : PnX-SI/GeoNature@652a7fd7b )

L'état de la PR est pour le moment en draft , car plusieurs soucis doivent être résolus :

Material Angular :
- le composant btn-select utilise des mat-chip-list et mat-chip renvoie des erreurs --> changement en mat-chip-grid et mat-chip-row (commit d3f3f43)
- les tabs de navigation nav-tab , nav-link ne sont plus utilisés et doivent être remplacé par mat-tab (voor composant monitoring-properties
- Les boutons ont été changés (voir détails des chanements à réaliser dans l'issue [style] Frontend : Changer les style "nav" ,"tab" , "button" pour être conforme à la dernière version GN naturalsolutions/gn_module_monitoring#77)
Détails des propriété des sites (composant monitoring-datatable , exemple l'ensemble des stations d'un circuit pour le module cheveches groupe de site):
- l'affichage de la liste des objets enfants apparait puis disparait --> source du problème trouvée : lors de l'entrée par protocoles c'est le fichier site.json du protcole qui est chargé , tandis que lors de l'entrée par site c'est le fichier site.Json du dossier generci qui est chargé . Les différences entre ces deux fichiers entraine des problèmes d'affichage notamment sur les filtres. ⚠️ Important voir comment uniformiser l'ajout et l'édition de site entre les deux entrées :
  - Plusieurs possibilités :
    - Checker les différences entre les champs "specific" / additionnels des deux fichiers de configs pour ne pas avertr / corriger ces différences ?
    - Ignorer les champs additionnels du fichier site.json du protocole et laisser la responsabilité d'affichage des champs additionnels liés aux types de site (associés au protocole en question)
Affichage des géométries:
- les géométries des objets enfants (exemple l'ensemble des géométries stations pour le module cheveches groupe de site , ne s'affiche pas sur la carte) --> c'était lié au prolbème des différences de fichiers json site.json (voir explication ci dessus)
: Affichage des visites liées à un site (erreur sur les droits): voir issue [fix] Erreur de chargement de droits lors du chargement des visites lié à un site (entrée protocole) naturalsolutions/gn_module_monitoring#76 (Normalement OK dans les derniers commit de la branche develop de monitoring)

Retours sur le point réalisé avec @amandine-sahl et @joelclems :

Entrée par protocole: pouvoir à l'édition et création de site renseigner les types de site associés au protocole (voir l'utilisation de data_utils pour récupérer les types de site associés au module et insérer à la volé les champs specific + ajouter des attributs "hidden" en fonction du type sélectionné
Entrée par site : Rendre la possibilité d'ajouter un nouveau site sans groupe de site (mais laissé la possibilité d'ajouter le groupe de site une fois dans le formulaire)
Entrée par site : La géométrie n'est pas affichée à l'édition et à la visualisation d'un site
Rendre possible le filtre/sort du tableau sur le champ "type de site"
La liste des observers est renseigné dans le fichier de config du module moitoring et c'est celle ci qui est chargé par défault
L'enchainement des saisies avec des champs "keep" reste fonctionnel

⚠️ Le commit ba1a112 concerne la revision fc90d31c677fpour laquelle il a été ajouté des droits 'CUD' pour le module monitoring afin de pouvoir ajouter des types de site via le module Admin

amandine-sahl

Première relecture des révisions

amandine-sahl · 2023-07-25T14:14:55Z

backend/gn_module_monitoring/migrations/6673266fb79c_remove_id_module_from_sites_complements.py

+    )
+    op.execute(statement)
+
+    # Drop column id_module


Si il y a des modules installées avec des vues d'export qui utilisent id_module
sqlalchemy.exc.InternalError: (psycopg2.errors.DependentObjectsStillExist) cannot drop column id_module of table gn_monitoring.t_site_complements because other objects depend on it
A voir faire une note de version où gérer automatiquement la suppression des vues

Ok ça marche je me le note dans un fichier de doc (en mettant note de version)

amandine-sahl · 2023-07-25T14:23:38Z

backend/gn_module_monitoring/migrations/b53bafb13ce8_create_bib_type_site.py

+        "bib_type_site",
+        "Table de définition des champs associés aux types de sites",
+        schema=monitorings_schema,
+    )


Peupler la table avec les valeurs déjà présentes dans t_base_sites

INSERT INTO gn_monitoring.bib_type_site AS bts (id_nomenclature_type_site) SELECT DISTINCT id_nomenclature_type_site FROM gn_monitoring.t_base_sites AS tbs ;

amandine-sahl · 2023-07-25T14:25:42Z

backend/gn_module_monitoring/migrations/ce54ba49ce5c_create_cor_type_site.py

+        "cor_type_site",
+        "Table d'association entre les sites et les types de sites",
+        schema=monitorings_schema,
+    )


Peupler les données

INSERT INTO gn_monitoring.cor_type_site SELECT id_nomenclature_type_site , id_base_site d FROM gn_monitoring.t_base_sites ;

amandine-sahl · 2023-07-25T14:33:33Z

backend/gn_module_monitoring/migrations/a54bafb13ce8_create_cor_module_type.py

+            nullable=False,
+        ),
+        sa.PrimaryKeyConstraint("id_type_site", "id_module", name="pk_cor_module_type"),
+        schema=monitorings_schema,


upgrade data

INSERT INTO gn_monitoring.cor_module_type (id_module, id_type_site ) SELECT DISTINCT csm.id_module, cts.id_type_site FROM gn_monitoring.cor_site_module AS csm JOIN gn_monitoring.cor_type_site AS cts ON Cts.id_base_site = csm.id_base_site ;

backend/gn_module_monitoring/migrations/e78003460441_correction_t_observation_detail.py

amandine-sahl · 2023-09-26T16:34:42Z

backend/gn_module_monitoring/migrations/126aca9e5503_add_observers_txt_column_t_base_visit.py

@@ -0,0 +1,36 @@
+"""add_observers_txt_column_t_base_visit
+


Doit être réalisé dans GN2 concomitamment avec le model

Concernant cette revision observers_txt" , tu dis que c'est coté GN ? Ca serait donc une revision avec un "depends_on" sur la premièe révision de la branche alembic monitorings ?

amandine-sahl · 2023-10-05T08:57:55Z

backend/gn_module_monitoring/migrations/e2b66850b5ee_add_digitiser_to_t_sites_groups.py

+                name=f"fk_{table}_{column}",
+                onupdate="CASCADE",
+            ),
+            nullable=False,


la colonne ne peut avoir la contrainte NOT NULL car il y a des données saisies et il n'est pas possible à 100% de retrouver l'id_digitizer

amandine-sahl · 2023-10-05T08:58:50Z

backend/gn_module_monitoring/migrations/a5dce2633e4c_add_id_digitiser_to_t_observations.py

+                name=f"fk_{table}_{column}",
+                onupdate="CASCADE",
+            ),
+            nullable=False,


avant de rendre la colonne NOT NULL il faut peupler les données existantes à partir des visites

codecov · 2023-10-10T09:03:42Z

Welcome to Codecov 🎉

Once merged to your default branch, Codecov will compare your coverage reports and display the results in this comment.

Thanks for integrating Codecov - We've got you covered ☂️

amandine-sahl · 2023-10-10T09:18:19Z

backend/gn_module_monitoring/utils/routes.py

+    return query
+
+
+def geojson_query(subquery) -> bytes:


Cette fonction ne peut pas fonctionner avec postgis < 3.0.0. Hors actuellement debian 10 est toujours supporté avec postgresql 11 et postgis 2.5

commit 70b0139 Author: Maxime Vergez <[email protected]> Date: Tue Dec 6 15:22:56 2022 +0100 style: apply black commit 2badedf Author: Maxime Vergez <[email protected]> Date: Tue Dec 6 15:11:41 2022 +0100 chore(db): removed unused import commit 57d9b52 Author: Maxime Vergez <[email protected]> Date: Tue Dec 6 14:47:48 2022 +0100 feat(db): improved & finished downgrade func commit 211780f Author: Maxime Vergez <[email protected]> Date: Tue Dec 6 09:32:12 2022 +0100 feat(db): wip: add upgrade and began downgrade Downgrade migration need a little bit more work due to the use of sql alchemy and not plain text sql commit 0b5a3b8 Author: Maxime Vergez <[email protected]> Date: Mon Dec 5 11:40:32 2022 +0100 feat: packaged module

Squashed commit of the following: commit 2bdc22a Author: Maxime Vergez <[email protected]> Date: Fri Dec 9 14:59:31 2022 +0100 chore: changed json_schema into config commit 4279389 Author: Maxime Vergez <[email protected]> Date: Tue Dec 6 18:37:41 2022 +0100 feat(db): add foreign key from site to cat From t_base_sites to bib_categorie_site commit 5cb5dfa Author: Maxime Vergez <[email protected]> Date: Tue Dec 6 16:47:27 2022 +0100 feat(api): add admin interface commit 559a0ec Author: Maxime Vergez <[email protected]> Date: Tue Dec 6 16:46:27 2022 +0100 feat(db): add bib_categorie_site table Add migration and model

…t description #2 (#15) * feat: [6.2] Page d'accueil modul monitoring layout et config title et description #2 * chore(config): applied black & removed imports

* feat(api): wip began add site routes + tests With site categories Also add tests * feat(api): add more routes * test(api): add tests and fixtures * style(api): applied black * feat(db): add migration to remove id_module Column in t_sites_groups * refactor(api): move utils for routes from sites * feat(api): wip: add sites groups route * test(api): wip: begin adding fixture site_groups * fix: remove id_module in all models * chore: rename route for better consistency * tests: moved site_groups in tests and add tests * chore(api): applied black * refactor(api): add filter params function And refact routes to use it

* feat(api): add association table with alembic Add model in backend and alembic migration Reviewed-by: andriac [Refs ticket]: #3 * test: WIP add test to see new relationship Adding test to see if categories are showing up when we call module Reviewed-by: andriacap [Refs ticket]: #3 * feat: add type site - categorie relation WIP - add selectfield to get type site in admin module Reviewed-by: andriac [Refs ticket]: #3 * feat(api): Flask admin and routes categories Clean code for change label list and form selectfield for the BibCategorieView in Flask Admin Add utils routes to get all subtable relationship in order to get back the label type site Review-by: andriac [Refs ticket]: #3 * refactor: remove paginate_nested For depth in as_dict() * test: fix tests due to as_dict depth * style: applied black and isort * chore: remove unused import Co-authored-by: Andria Capai <[email protected]>

* tests: make tests and fixtures work for modules By changing the way a monitoring module is created in the fixture Add two tests to check the relationship * feat(api): add categories in edit module * style: applied black

Need to set the current working directory of pytest the directory where the gn_monitoring_module repo is (so with the frontend etc.)

clean: Merge la PR move test_route Co-authored-by: Andria Capai <[email protected]>

* feat(api): remove id_module from sites_complements Co-authored-by: andriacap <[email protected]> * feat(api): create schema for sites_groups * test: add sites_group to site fixture * test: wip add test for sites_groups schemas * chore(api): remove depth parameter from paginate * test: updated to work with sites_group schema * feat: categorie site with marshmallow Test and marshmallow create/refactor to adapt for bibcategorie site paginate WIP : Adapt load_only site_type in test to "assert" same object when initiate BibCategorieSite [Refs ticket]: #3 * feat(api): route /sites/categories/id with schema Changing the route to return a dump Marshmallow schema BibCategorieSitesSchema Reviewed-by: andriac [Refs ticket]: #3 * test(api): routes get categoires label Change the "as_dict" by schema.dump in order to use the Marshmallow schema created Reviewed-by: andriac [Refs ticket]: #3 * feat(api): Sites: cols to geoserializable + schema * style(api): applied black * test(api): add test for Site Schema * style(api): applied black to test_site * refactor(api): instantiate schema once Instead of for each all() iteration * chore(api): remove useless comments * chore(api): remove useless comments and imports Co-authored-by: andriacap <[email protected]> Co-authored-by: Andria Capai <[email protected]>

To replace it with a bib nomenclature type: TYPE_SITE

* fix(api): add int conversion for limit/offset * test(api): add test for get_limit_offset * fix(api): max_per_page => per_page & test * test(api): update test with changes on schema Since marshmallow schemas, the json returned by pagination has changed for items and not sites

* feat(api): add Query Class to sites, grps & cats * feat(api): add sort and fix _get_model Via _get_entity * test(api): test sort query * feat(api): add api sort/sort_dir params To be able to sort through REST Api * fix(api): check if integer to avoid using ilike In filter_by_params * test(api): add test to check filter integer

* Merged feat/package-module-alembic commit 70b0139 Author: Maxime Vergez <[email protected]> Date: Tue Dec 6 15:22:56 2022 +0100 style: apply black commit 2badedf Author: Maxime Vergez <[email protected]> Date: Tue Dec 6 15:11:41 2022 +0100 chore(db): removed unused import commit 57d9b52 Author: Maxime Vergez <[email protected]> Date: Tue Dec 6 14:47:48 2022 +0100 feat(db): improved & finished downgrade func commit 211780f Author: Maxime Vergez <[email protected]> Date: Tue Dec 6 09:32:12 2022 +0100 feat(db): wip: add upgrade and began downgrade Downgrade migration need a little bit more work due to the use of sql alchemy and not plain text sql commit 0b5a3b8 Author: Maxime Vergez <[email protected]> Date: Mon Dec 5 11:40:32 2022 +0100 feat: packaged module * Feat/edit categories module (#19) * tests: make tests and fixtures work for modules By changing the way a monitoring module is created in the fixture Add two tests to check the relationship * feat(api): add categories in edit module * style: applied black * fix(config): changed categories into items Following the change with marshmallow schemas

Basé sur les retours de Camille, Amandine et MNHN Reviewed-by: andriac

Reviewed-by: andriacap

Reset specific field when quit form component (entrance through site) Reviewed-by: andriac

Load permission on 'ALL' for modules error . Change to precise object visit permission Reviewed-by: andriac

Reviewed-by: andriacap

Reviewed-by: andriac

Manage geometry site group (load, reload, on cancel, manage layer etc) (entrance through site) Reviewed-by: andriac

Allow user to create and edit geometry for site group Case : - site group with child --> if no geom created, geometry is calculated --> if geom created you can edit this geom - site group without child --> geom can be null (otherwise it's possible to create geom) - Remove console.log Reviewed-by: andriacap

Add message about geometry site group to infomr user that the current geometry is calculated based on site group's child Reviewed-by: andriacap

Error on check id_organism compare to organism actor Reviewed-by: andriacap

Creation d'un nouveau fichier pour expliquer l'entrée par groupe de site et site Modification de la présentation du module monitoring Modification du paragraphe de Permissions Ajout d'images descriptives Reviewed-by: andriacap

Add part to explain link between type site field in edit and creation site form Reviewed-by: andriacap

- Fix bug changeActiveTab - Add displaying conditional site group geometry through protocol Reviewed-by: andriacap

Reviewed-by: andriacap

Error when using docker build Reviewed-by: andriacap

Reviewed-by: andriacap

joelclems changed the base branch from dev-suivi-eolien to develop July 24, 2023 13:47

andriacap mentioned this pull request Jul 27, 2023

[dev] Rebase la branche dev-suivi-eolien avec la dev de monitoring naturalsolutions/gn_module_monitoring#52

Closed

andriacap force-pushed the dev-suivi-eolien-rebased branch 3 times, most recently from 5be029d to ae84016 Compare July 28, 2023 12:02

andriacap force-pushed the dev-suivi-eolien-rebased branch 2 times, most recently from 0a4bf79 to 672d184 Compare August 23, 2023 09:19

andriacap force-pushed the dev-suivi-eolien-rebased branch 2 times, most recently from 2850df2 to 1738b78 Compare September 7, 2023 16:17

amandine-sahl reviewed Sep 26, 2023

View reviewed changes

amandine-sahl force-pushed the dev-suivi-eolien-rebased branch from 412f459 to 9906bf8 Compare October 5, 2023 08:24

amandine-sahl reviewed Oct 5, 2023

View reviewed changes

amandine-sahl force-pushed the dev-suivi-eolien-rebased branch from 6c544bc to 462d408 Compare October 10, 2023 07:43

amandine-sahl reviewed Oct 10, 2023

View reviewed changes

Maxime Vergez and others added 14 commits October 10, 2023 11:41

feat: [6.2] Page d'accueil module monitoring layout et config title e…

b9a23a4

…t description #2 (#15) * feat: [6.2] Page d'accueil modul monitoring layout et config title et description #2 * chore(config): applied black & removed imports

Feat/edit categories module (#19)

94ef8f9

* tests: make tests and fixtures work for modules By changing the way a monitoring module is created in the fixture Add two tests to check the relationship * feat(api): add categories in edit module * style: applied black

test: refactor fixtures to load them automatically (#20)

e963eac

Need to set the current working directory of pytest the directory where the gn_monitoring_module repo is (so with the frontend etc.)

test: move test_route in parent dir (#17)

47d1b7e

clean: Merge la PR move test_route Co-authored-by: Andria Capai <[email protected]>

refactor(api): remove id_type in admin (#22)

7a1d9f0

To replace it with a bib nomenclature type: TYPE_SITE

style(config): rename attribut label of categories (#23)

df02c69

amandine-sahl added 4 commits October 10, 2023 11:41

Correction tests

c34c3e1

Black

9168c5c

pytest install monitoring module

90106b5

Test if migration is run during installation or migration

d3a81d2

amandine-sahl force-pushed the dev-suivi-eolien-rebased branch from e2f21c9 to d3a81d2 Compare October 10, 2023 09:41

andriacap and others added 19 commits October 10, 2023 16:51

refact: load all types site from btn-select comp

ec84328

Basé sur les retours de Camille, Amandine et MNHN Reviewed-by: andriac

style: apply lint frontend

8bd3555

Reviewed-by: andriacap

fix: specific field kept in form

429213c

Reset specific field when quit form component (entrance through site) Reviewed-by: andriac

hotfix: problem create visit

0710599

Load permission on 'ALL' for modules error . Change to precise object visit permission Reviewed-by: andriac

style: centered protocol home page

38167e5

Reviewed-by: andriacap

wip: manage geometry site group

3ae0602

Reviewed-by: andriac

fix:multiple layer and geom site group

086568f

Manage geometry site group (load, reload, on cancel, manage layer etc) (entrance through site) Reviewed-by: andriac

ux: add message to inform about geom site group

a15b576

Add message about geometry site group to infomr user that the current geometry is calculated based on site group's child Reviewed-by: andriacap

lint: backend

31bafa2

fix: problem to read data same organism

9f7f021

Error on check id_organism compare to organism actor Reviewed-by: andriacap

docs: add documentation for new features (wip)

6c85ff6

Creation d'un nouveau fichier pour expliquer l'entrée par groupe de site et site Modification de la présentation du module monitoring Modification du paragraphe de Permissions Ajout d'images descriptives Reviewed-by: andriacap

docs: add description type site form

3576127

Add part to explain link between type site field in edit and creation site form Reviewed-by: andriacap

Optimize model and serialization

2a06f67

Use marshmallow serializer

980bccc

feat: manage geom through protocol

223df5b

- Fix bug changeActiveTab - Add displaying conditional site group geometry through protocol Reviewed-by: andriacap

fix: create site group without geom

7dc2acd

Reviewed-by: andriacap

hotfix: remove import AppConfig

e044a6e

Error when using docker build Reviewed-by: andriacap

docs: add observers_txt in "sous_module"

2e1096d

Reviewed-by: andriacap

amandine-sahl mentioned this pull request Dec 8, 2023

[DRAFT][Permissions]Ajout des filtres sur la portée des données #263

Closed

amandine-sahl changed the base branch from develop to dev-suivi-eolien December 8, 2023 11:05

amandine-sahl force-pushed the dev-suivi-eolien branch from 606d08c to 9e9878e Compare December 8, 2023 11:13

Resolve conflicts

3096589

amandine-sahl marked this pull request as ready for review December 8, 2023 11:22

amandine-sahl merged commit c710880 into PnX-SI:dev-suivi-eolien Dec 8, 2023
2 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Dev suivi eolien rebased on develop #238

Dev suivi eolien rebased on develop #238

andriacap commented Jul 24, 2023 •

edited by jacquesfize

Loading

amandine-sahl left a comment

amandine-sahl Jul 25, 2023

andriacap Sep 29, 2023

amandine-sahl Jul 25, 2023

amandine-sahl Jul 25, 2023

amandine-sahl Jul 25, 2023

amandine-sahl Sep 26, 2023

andriacap Sep 27, 2023

amandine-sahl Oct 5, 2023

amandine-sahl Oct 5, 2023

codecov bot commented Oct 10, 2023

amandine-sahl Oct 10, 2023 •

edited

Loading

Dev suivi eolien rebased on develop #238

Dev suivi eolien rebased on develop #238

Conversation

andriacap commented Jul 24, 2023 • edited by jacquesfize Loading

amandine-sahl left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

codecov bot commented Oct 10, 2023

Welcome to Codecov 🎉

amandine-sahl Oct 10, 2023 • edited Loading

Choose a reason for hiding this comment

andriacap commented Jul 24, 2023 •

edited by jacquesfize

Loading

amandine-sahl Oct 10, 2023 •

edited

Loading