Skip to content

Commit

Permalink
Added some workflow stuff, ported make_env.py, in progres...
Browse files Browse the repository at this point in the history
  • Loading branch information
dalmijn committed Aug 11, 2023
1 parent 9877fb4 commit 542aec9
Show file tree
Hide file tree
Showing 8 changed files with 307 additions and 6 deletions.
49 changes: 49 additions & 0 deletions .github/ISSUE_TEMPLATE/bug_report.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,49 @@
---
name: Bug Report
description: Report incorrect behavior in the HydroMT library
labels: [Bug, Needs refinement]

body:
- type: checkboxes
id: checks
attributes:
label: HydroMT version checks
options:
- label: I have checked that this issue has not already been reported.
required: true
- label: I have checked that this bug exists on the latest version of HydroMT.
required: true
- type: textarea
id: example
attributes:
description: >
Please provide a minimal, copy-pastable example or a link to a public repository that reproduces the behavior. If providing a copy pastable example,
you may assume your in a clean up to date version of hydromt with a python enviroment active. In the case of a repository, ensure the repository
has a README.md which includes intructions to reproduce the behaviour.
label: Reproducible Example
validations:
required: true
- type: textarea
id: current-behaviour
attributes:
description: >
Please provide a description of the incorrect behaviour shown in the reproducible example
label: Current behaviour
validations:
required: true
- type: textarea
id: expected-behaviour
attributes:
description: >
Please provide a description of what you think the behaviour should be
label: Desired behaviour
validations:
required: true
- type: textarea
id: additional
attributes:
description: >
Please add any other context about the bug here
label: Additional context
validations:
required: false
7 changes: 7 additions & 0 deletions .github/ISSUE_TEMPLATE/config.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
---

blank_issues_enabled: false
contact_links:
- name: Ask a question
url: https://github.com/Deltares/hydromt/discussions
about: Ask questions and discuss with other community members
45 changes: 45 additions & 0 deletions .github/ISSUE_TEMPLATE/documentation_improvement.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,45 @@
---
name: Documentation Improvement
description: Report wrong or missing documentation
labels: [Documentation, Needs refinement]

body:
- type: checkboxes
attributes:
label: HydroMT version checks
options:
- label: >
I have checked that the issue still exists on the latest versions of the docs
on `main` [here](https://github.com/Deltares/hydromt)
required: true
- type: dropdown
id: kind
attributes:
description: What kind of documentation issue is this?
label: Kind of issue
options:
- Docs are wrong
- Docs are unclear
- Docs are missing
validations:
required: true
- type: textarea
id: location
attributes:
description: >
If the docs are wrong or unclear please provide the URL of the documentation in question
label: Location of the documentation
- type: textarea
id: problem
attributes:
description: >
Please provide a description of the documentation problem
label: Documentation problem
validations:
required: true
- type: textarea
id: suggested-fix
attributes:
description: >
Please explain your suggested fix and why it's better than the existing documentation
label: Suggested fix for documentation
35 changes: 35 additions & 0 deletions .github/ISSUE_TEMPLATE/feature_request.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,35 @@
---
name: Feature Request
description: Suggest an idea/enhancement for HydroMT
labels: [Enhancement, Needs refinement]

body:
- type: dropdown
id: checks
attributes:
description: What kind of feature request is this?
label: Kind of request
options:
- Adding new functionality
- Changing existing functionality
- Removing existing functionality
- type: textarea
id: description
attributes:
description: >
Please provide a clear and concise description of the feature you're requesting
label: Enhancement Description
validations:
required: true
- type: textarea
id: use-case
attributes:
description: >
Please describe a situation in which this feature would be useful to you, with code or cli examples if possible
label: Use case
- type: textarea
id: context
attributes:
description: >
Please add any other context about the enhancement here
label: Additional Context
14 changes: 14 additions & 0 deletions .github/pull_request_template.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
## Issue addressed
Fixes #<issue number>

## Explanation
Explain how you addressed the bug/feature request, what choices you made and why.

## Checklist
- [ ] Updated tests or added new tests
- [ ] Branch is up to date with `main`
- [ ] Tests & pre-commit hooks pass
- [ ] Updated documentation if needed

## Additional Notes (optional)
Add any additional notes or information that may be helpful.
4 changes: 4 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,9 @@ __pycache__/
*.py[cod]
*$py.class

# Temporary files
tmp/

# C extensions
*.so

Expand Down Expand Up @@ -109,6 +112,7 @@ venv/
ENV/
env.bak/
venv.bak/
environment.yml

# Spyder project settings
.spyderproject
Expand Down
113 changes: 113 additions & 0 deletions make_env.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,113 @@
"""A simple script to generate enviroment.yml files from pyproject.toml."""

import argparse
import re
from sys import version_info
from typing import List

if version_info.minor >= 11:
from tomllib import load
else:
from tomli import load


# our quick and dirty implementation of recursive depedencies
def _parse_profile(profile_str: str, opt_deps: dict, project_name: str) -> List[str]:
if profile_str is None or profile_str == "":
return []

pat = re.compile(r"\s*" + project_name + r"\[(.*)\]\s*")
parsed = []
queue = [f"{project_name}[{x.strip()}]" for x in profile_str.split(",")]
while len(queue) > 0:
dep = queue.pop(0)
if dep == "":
continue
m = pat.match(dep)
if m:
# if we match the patern, all list elts have to be dependenciy groups
dep_groups = [d.strip() for d in m.groups(0)[0].split(",")]
unknown_dep_groups = set(dep_groups) - set(opt_deps.keys())
if len(unknown_dep_groups) > 0:
raise RuntimeError(f"unknown dependency group(s): {unknown_dep_groups}")
queue.extend(dep_groups)
continue

if dep in opt_deps:
queue.extend([x.strip() for x in opt_deps[dep]])
else:
parsed.append(dep)

return parsed


parser = argparse.ArgumentParser()

parser.add_argument("profile", default="dev,test", nargs="?")
parser.add_argument("--output", "-o", default="environment.yml")
parser.add_argument("--channels", "-c", default=None)
parser.add_argument("--name", "-n", default=None)
parser.add_argument("--py-version", "-p", default=None)
args = parser.parse_args()

#
with open("pyproject.toml", "rb") as f:
toml = load(f)
deps = toml["project"]["dependencies"]
opt_deps = toml["project"]["optional-dependencies"]
project_name = toml["project"]["name"]
# specific conda_install settings
install_config = toml["tool"].get("make_env", {})
deps_not_in_conda = install_config.get("deps_not_in_conda", [])
channels = install_config.get("channels", ["conda-forge"])
if args.channels is not None:
channels.extend(args.channels.split(","))

# parse environment name
name = args.name
if name is None:
name = project_name
print(f"Environment name: {name}")

# parse dependencies groups and flavours
# "min" equals no optional dependencies
deps_to_install = deps.copy()
if args.profile not in ["", "min"]:
extra_deps = _parse_profile(args.profile, opt_deps, project_name)
deps_to_install.extend(extra_deps)

conda_deps = []
pip_deps = []
for dep in deps_to_install:
if dep in deps_not_in_conda:
pip_deps.append(dep)
else:
conda_deps.append(dep)
if args.py_version is not None:
conda_deps.append(f"python=={args.py_version}")

# add pip as a conda dependency if we have pip deps
if len(pip_deps) > 0:
conda_deps.append("pip")

# the list(set()) is to remove duplicates
conda_deps_to_install_string = "\n- ".join(sorted(list(set(conda_deps))))
channels_string = "\n- ".join(set(channels))

# create environment.yml
env_spec = f"""name: {name}
channels:
- {channels_string}
dependencies:
- {conda_deps_to_install_string}
"""
if len(pip_deps) > 0:
pip_deps_to_install_string = "\n - ".join(sorted(list(set(pip_deps))))
env_spec += f"""- pip:
- {pip_deps_to_install_string}
"""

with open(args.output, "w") as out:
out.write(env_spec)
46 changes: 40 additions & 6 deletions pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -10,8 +10,9 @@ authors = [
]
dependencies = [
"dask",
"geopandas >=0.10",
"hydromt >=0.8",
"geopandas>=0.10",
"hydroengine",
"hydromt>=0.8",
"netcdf4",
"numpy",
"pandas",
Expand All @@ -23,10 +24,9 @@ dependencies = [
"toml",
"xarray",
]
requires-python = ">=3.8" # fix tests to support older versions
requires-python = ">=3.9"
readme = "README.rst"
classifiers = [
# https://pypi.python.org/pypi?%3Aaction=list_classifiers
"Development Status :: 4 - Beta",
"Intended Audience :: Developers",
"Intended Audience :: Science/Research",
Expand All @@ -37,14 +37,38 @@ classifiers = [
dynamic = ['version', 'description']

[project.optional-dependencies]
test = ["pytest>=2.7.3", "pytest-cov", "black"]
io = [
"requests",
]
extra = [
"gwwapi",
"pyet",
]
dev = [
"black",
"pip>=23.1.2",
]
test = [
"pytest>=2.7.3",
"pytest-cov",
"pytest-mock",
]
doc = [
"nbsphinx",
"sphinx",
"sphinx_design",
"pydata-sphinx-theme",
"sphinx_autosummary_accessors",
]
examples = [
"jupyterlab",
"cartopy",
"matplotlib-base",
"notebook",
]

full = ["hydromt_wflow[io,extra,dev,test,doc,examples]"]
slim = ["hydromt_wflow[io,extra,examples]"]

[project.urls]
Documentation = "http://deltares.github.io/hydromt_wflow/latest/"
Expand All @@ -54,9 +78,19 @@ Source = "https://github.com/Deltares/hydromt_wflow"
wflow = "hydromt_wflow.wflow:WflowModel"
wflow_sediment = "hydromt_wflow.wflow_sediment:WflowSedimentModel"

[tool.make_env]
channels = ["conda-forge"]
deps_not_in_conda = [
"gwwapi",
"hydroengine",
"sphinx_autosummary_accessors",
"sphinx_design",
"pyet",
]

[tool.black]
line-length = 88
target-version = ['py38']
target-version = ['py39']

[tool.flit.sdist]
include = ["hydromt_wflow"]
Expand Down

0 comments on commit 542aec9

Please sign in to comment.