Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Dev #175

Merged
merged 13 commits into from
Apr 8, 2024
Merged

Dev #175

Show file tree
Hide file tree
Changes from 6 commits
Commits
Show all changes
13 commits
Select commit Hold shift + click to select a range
08d89d8
Update README.md
brey Jan 17, 2024
2a6db66
Add hotstart option 1 in cast
brey Jan 17, 2024
1fe314c
remove tg metadata from simulation station output
brey Jan 21, 2024
1dd50cc
resolve #173
brey Jan 21, 2024
730d342
add skiprows argument to reading station output
brey Jan 21, 2024
cf32241
Deal with schism bug in station output
brey Jan 21, 2024
b70acbf
Update mesh.py
brey Apr 6, 2024
98120ed
towards adding a tiling feature
brey Apr 6, 2024
4378335
black update
brey Apr 6, 2024
f07267e
ci: Relax black constraint
pmav99 Apr 8, 2024
a9633aa
ci: Run tests on 3.12 too
pmav99 Apr 8, 2024
6a840b7
schism: Add `parse_mirror_out()`
pmav99 Apr 8, 2024
e48e864
tests: Fix failings tests
pmav99 Apr 8, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
Framework for Sea Level Hydrodynamic simulations
================================================

[![Documentation Status](https://readthedocs.org/projects/pyposeidon/badge/?version=latest)](https://pyposeidon.readthedocs.io/en/latest/?badge=latest) ![GitHub release (latest by date)](https://img.shields.io/github/v/release/ec-jrc/pyPoseidon) ![CI](https://github.com/ec-jrc/pyPoseidon/actions/workflows/conda_pip.yml/badge.svg) ![CI](https://github.com/ec-jrc/pyPoseidon/actions/workflows/conda_only.yml/badge.svg) ![CI](https://github.com/ec-jrc/pyPoseidon/actions/workflows/code_quality.yml/badge.svg) [![Binder](https://mybinder.org/badge_logo.svg)](https://mybinder.org/v2/gh/ec-jrc/pyPoseidon/master?urlpath=%2Flab)
[![Documentation Status](https://readthedocs.org/projects/pyposeidon/badge/?version=latest)](https://pyposeidon.readthedocs.io/en/latest/?badge=latest) ![GitHub release (latest by date)](https://img.shields.io/github/v/release/ec-jrc/pyPoseidon) ![CI](https://github.com/ec-jrc/pyPoseidon/actions/workflows/conda_pip.yml/badge.svg) [![Binder](https://mybinder.org/badge_logo.svg)](https://mybinder.org/v2/gh/ec-jrc/pyPoseidon/master?urlpath=%2Flab)

This is a development project utilising multiple solvers (currently `DELFT3D` & `SCHISM`) for simulating sea level height (currently only storm surge). The purpose is to create a simple, portable and transparent way of setting up, running and analysing hydrodynamic computations through python scripts and Jupyter Notebooks (http://jupyter.org). See Notebooks in Tutorial/ for relevant prototypes.

Expand Down
22 changes: 16 additions & 6 deletions pyposeidon/schism.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1819,11 +1819,11 @@ def set_obs(self, **kwargs):
stations.index += 1
stations["gindex"] = mesh_index
try:
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would even suppress assigning self.obs (above here line 1755: self.obs = tg_database)
otherwise it gets written to <module>_model.json

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

with the new implementation tg_database (and thus self.obs) is always a filename, either originally or after using searvey,

            tg_database = os.path.join(path, "stations.json")
            tg.to_file(tg_database)
        self.obs = tg_database

stations["location"] = self.obs.location.values
stations["provider_id"] = self.obs.ioc_code.values
stations["location"] = tgn.location.values
stations["provider_id"] = tgn.ioc_code.values
stations["provider"] = "ioc"
stations["longitude"] = self.obs.longitude.values
stations["latitude"] = self.obs.latitude.values
stations["longitude"] = tgn.longitude.values
stations["latitude"] = tgn.latitude.values
except:
pass

Expand Down Expand Up @@ -1901,14 +1901,24 @@ def get_station_sim_data(self, **kwargs):

dstamp = kwargs.get("dstamp", self.rdate)

skiprows = kwargs.get("station_skiprows", 0)

try:
dt = self.parameters["dt"]
except:
dt = self.params["core"]["dt"]

dfs = []
for idx in vals.index:
obs = np.loadtxt(sfiles[idx])
obs = np.loadtxt(sfiles[idx], skiprows=skiprows)
df = pd.DataFrame(obs)
df = df.set_index(0)
df.index.name = "time"
df.columns.name = vals.loc[idx, "variable"]
df.index = pd.to_datetime(dstamp) + pd.to_timedelta(df.index, unit="S")
# deal with schism bug
ns = np.arange(1, df.shape[0] + 1)
df.index = pd.to_datetime(dstamp) + pd.to_timedelta(ns * dt, unit="S")
# df.index = pd.to_datetime(dstamp) + pd.to_timedelta(df.index, unit="S")
pindex = pd.MultiIndex.from_product([df.T.columns, df.T.index])

r = pd.DataFrame(df.values.flatten(), index=pindex, columns=[vals.loc[idx, "variable"]])
Expand Down
48 changes: 32 additions & 16 deletions pyposeidon/utils/cast.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -279,6 +279,8 @@ def run(self, **kwargs):

copy = get_value(self, kwargs, "copy", False)

ihot = get_value(self, kwargs, "ihot", 1)

pwd = os.getcwd()

self.origin = self.model.rpath
Expand Down Expand Up @@ -336,7 +338,6 @@ def run(self, **kwargs):
info["config_file"] = os.path.join(ppath, "param.nml")

# update the properties

info["rdate"] = self.rdate
info["start_date"] = self.sdate
info["time_frame"] = self.time_frame
Expand All @@ -362,7 +363,10 @@ def run(self, **kwargs):
logger.debug("create restart file")

# check for combine hotstart
hotout = int((self.sdate - self.rdate).total_seconds() / info["params"]["core"]["dt"])
if ihot == 2:
hotout = int((self.sdate - self.rdate).total_seconds() / info["params"]["core"]["dt"])
elif ihot == 1:
hotout = self.parameters["nhot_write"]
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You should (almost) always add an else clase to an if/elif.

What if the user adds an invalid ihot value? Then hotout will not be defined and you will get a NameError which will confusing.

if instead we have:

else:
    raise ValueError("Acceptable values for `ihot` are 1 and 2, not: %s", ihot)

it will be immediately apparent what the issue is.

For the record, ideally, we should be validating the user input as soon as we get it, but to do that properly will require a major restructure so let's keep it simple for now.

logger.debug("hotout_it = {}".format(hotout))

# link restart file
Expand Down Expand Up @@ -430,20 +434,32 @@ def run(self, **kwargs):
logger.warning("meteo files present\n")

# modify param file
rnday_new = (self.sdate - self.rdate).total_seconds() / (3600 * 24.0) + pd.to_timedelta(
self.time_frame
).total_seconds() / (3600 * 24.0)
hotout_write = int(rnday_new * 24 * 3600 / info["params"]["core"]["dt"])
info["parameters"].update(
{
"ihot": 2,
"rnday": rnday_new,
"start_hour": self.rdate.hour,
"start_day": self.rdate.day,
"start_month": self.rdate.month,
"start_year": self.rdate.year,
}
)
if ihot == 2:
rnday_new = (self.sdate - self.rdate).total_seconds() / (3600 * 24.0) + pd.to_timedelta(
self.time_frame
).total_seconds() / (3600 * 24.0)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The formatting is ugly here. Either introduce new variables or maybe try some variation of this (I didn't test it):

            rnday_new = (self.sdate - self.rdate) + pd.to_timedelta(self.time_frame)
            rnday_new = rnday_new.total_seconds() / (3600 * 24.0)
            ...

hotout_write = int(rnday_new * 24 * 3600 / info["params"]["core"]["dt"])
info["parameters"].update(
{
"ihot": 2,
"rnday": rnday_new,
"start_hour": self.rdate.hour,
"start_day": self.rdate.day,
"start_month": self.rdate.month,
"start_year": self.rdate.year,
}
)
elif ihot == 1:
info["parameters"].update(
{
"ihot": 1,
"start_hour": self.sdate.hour,
"start_day": self.sdate.day,
"start_month": self.sdate.month,
"start_year": self.sdate.year,
}
)
# else:

m.config(output=True, **info) # save param.nml

Expand Down
3 changes: 2 additions & 1 deletion pyposeidon/utils/post.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -239,7 +239,8 @@ def to_thalassa(folder, **kwargs):

output_path = os.path.join(rpath, filename)

vdata.to_netcdf(output_path)
# output sim data
vdata[["node", "ioc_code", "lat", "lon", "location", "elev_sim"]].to_netcdf(output_path)
logger.info(f"..done with {filename} file\n")


Expand Down
Loading