Yamale error validating data with schema for xscen conversion

[juliettelavoie]

Setup Information

• Xclim version: xclim-0.47.4b0
• Python version: 3.11.6
• Operating System: linux

more context:
I had an existing branch that worked well. I wanted to update xclim to @coxipi's branch npdf_gpies.

pip uninstall xclim
 pip install git+https://github.com/Ouranosinc/xclim.git@npdf_gpies
# try to run code, get error module yamale not found
mamba install yamale

Description

I used xscen.search_data_catalog to create dtr (using https://github.com/Ouranosinc/xscen/blob/main/xscen/xclim_modules/conversions.yml), but I got this error.

Traceback (most recent call last):
  File "/expl7/jlavoie/.conda/envs/ic6-mbcn/lib/python3.11/site-packages/IPython/core/interactiveshell.py", line 3526, in run_code
    exec(code_obj, self.user_global_ns, self.user_ns)
  File "<ipython-input-2-d6a94190962a>", line 1, in <module>
    runfile('/home/jlavoie/projets/info-crue-cmip6/workflow.py', wdir='/home/jlavoie/projets/info-crue-cmip6')
  File "/home/jlavoie/.pycharm_helpers/pydev/_pydev_bundle/pydev_umd.py", line 198, in runfile
    pydev_imports.execfile(filename, global_vars, local_vars)  # execute the script
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/jlavoie/.pycharm_helpers/pydev/_pydev_imps/_pydev_execfile.py", line 18, in execfile
    exec(compile(contents+"\n", file, 'exec'), glob, loc)
  File "/home/jlavoie/projets/info-crue-cmip6/workflow.py", line 214, in <module>
    cat_sim = search_data_catalogs(
              ^^^^^^^^^^^^^^^^^^^^^
  File "/expl7/jlavoie/.conda/envs/ic6-mbcn/lib/python3.11/site-packages/xscen/config.py", line 217, in _wrapper
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/expl7/jlavoie/.conda/envs/ic6-mbcn/lib/python3.11/site-packages/xscen/extract.py", line 586, in search_data_catalogs
    "registry": registry_from_module(load_xclim_module(conversion_yaml))
                                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/expl7/jlavoie/.conda/envs/ic6-mbcn/lib/python3.11/site-packages/xscen/indicators.py", line 58, in load_xclim_module
    return xc.build_indicator_module_from_yaml(filename)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/expl7/jlavoie/.conda/envs/ic6-mbcn/lib/python3.11/site-packages/xclim/core/indicator.py", line 1724, in build_indicator_module_from_yaml
    yamale.validate(
  File "/expl7/jlavoie/.conda/envs/ic6-mbcn/lib/python3.11/site-packages/yamale/yamale.py", line 43, in validate
    raise YamaleError(results)
yamale.yamale_error.YamaleError: Error validating data with schema '/expl7/jlavoie/.conda/envs/ic6-mbcn/lib/python3.11/site-packages/xclim/data/schema.yml'
	indicators.potential_evapotranspiration.parameters.tasmin: '{'kind': 0}' is not a str.
	indicators.potential_evapotranspiration.parameters.tasmin: '{'kind': 0}' is not a num.
	indicators.potential_evapotranspiration.parameters.tasmin: '{'kind': 0}' is not a bool.
	indicators.potential_evapotranspiration.parameters.tasmin: '{'kind': 0}' is not a null.
	indicators.potential_evapotranspiration.parameters.tasmin.kind: Unexpected element
	indicators.potential_evapotranspiration.parameters.tasmax: '{'kind': 0}' is not a str.
	indicators.potential_evapotranspiration.parameters.tasmax: '{'kind': 0}' is not a num.
	indicators.potential_evapotranspiration.parameters.tasmax: '{'kind': 0}' is not a bool.
	indicators.potential_evapotranspiration.parameters.tasmax: '{'kind': 0}' is not a null.
	indicators.potential_evapotranspiration.parameters.tasmax.kind: Unexpected element
	indicators.dtr.src_freq: 'D' is not a list.
	indicators.tasmin_from_dtr.src_freq: 'D' is not a list.
	indicators.tasmax_from_dtr.src_freq: 'D' is not a list.

Steps To Reproduce

import xscen as xs
cat_sim= xs.search_data_catalogs(data_catalogs=['/tank/scenario/catalogues/simulation.json'],
                                 variables_and_freqs={'dtr':'D'},
                                 allow_conversion=True,
                                 other_search_criteria={'source':'CanESM5','experiment':'ssp585','processing_level':'raw'})

Additional context

Not sure, why this pops up now ?

Contribution

• I would be willing/able to open a Pull Request to address this bug.

Code of Conduct

• I agree to follow this project's Code of Conduct

[juliettelavoie]

@Zeitsperre I don't think you meant to close the issue ? The error I am getting is not fixed by adding the dependence, or am it missing something ?

[Zeitsperre]

@juliettelavoie My mistake! Yes, let's keep the issue open.

[juliettelavoie]

@coxipi To answer your slack question, I get the same error with the master (not only with your branch).

[aulemahal]

This issue was because of #1560. The package was used before only during the tests, that PR makes use of it each time a module is created. The full (dev) env file gets installed on the CI, so we didn't detect that it was missing from the actual requirements!

And some yaml formats are missing from the tests and schema!