Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Propose a space delimited UCI #171

Open
timcera opened this issue Jul 11, 2024 · 5 comments
Open

Propose a space delimited UCI #171

timcera opened this issue Jul 11, 2024 · 5 comments

Comments

@timcera
Copy link
Contributor

timcera commented Jul 11, 2024

The import_uci function takes a HSPF UCI file and imports the tables into the HDF5 file for hsp2 to run. Using this format and process though imposes the HSPF limits that don't exist in hsp2, the most constraining being the three digit limits on PERLND, IMPLND, and RCHRES identifiers. Proposing a space delimited UCI where an @ symbol is used to indicate a default.

HSPF UCI snippet:

  PWAT-PARM3
    <PLS >***
    # - #*** PETMAX PETMIN INFEXP INFILD DEEPFR BASETP AGWETP
    1   7                            1.6
    9            39     33    3.0    1.5
   10
  END PWAT-PARM3

Proposed space delimited UCI for hsp2 (note the ability to have very large PERLND ids):

  PWAT-PARM3
    <PLS >***
    # - #***     PETMAX PETMIN INFEXP INFILD DEEPFR BASETP AGWETP
    10100   10107   @   @     1.6    @    @   @
    90100   @    39     33    3.0    1.5  @  @  @
    9010     @    @    @    @    @    @   @
  END PWAT-PARM3

Maybe not need trailing "@" default symbols...

  PWAT-PARM3
    <PLS >***
    # - #***     PETMAX PETMIN INFEXP INFILD DEEPFR BASETP AGWETP
    10100   10107   @   @     1.6
    90100   @    39     33    3.0    1.5 
    9010
  END PWAT-PARM3
@timcera
Copy link
Contributor Author

timcera commented Jul 27, 2024

Thinking about this I think a better approach is to have a CSV file for each table because it would be easier to create with GIS or other external processes. The file name would be the name of the block. So something like:

PWAT-PARM3.csv

ID_FIRST,ID_LAST,PETMAX,PETMIN,INFEXP,INFILD,DEEPFR,BASETP,AGWETP
10100,10107, , ,1.6
90100, ,39,33,3.0,1.5

Empty fields would be filled with the default for that parameter. Thinking that the header row would be convenience for the user and the order of the parameters would have to match the table in the UCI.

This would work very well with parameter estimation with PEST.

Then have a new command...
hsp2 import_csvs *.csv new_model.h5

@rburghol
Copy link
Contributor

The folks at the ches bay program have long used CSVs for storing 100s of subwatersheds, similar to what you propose here.
I generally like the CSV from the standpoint of easy conversion to a queryable table, however, I'm also quite partial to the use of JSON, with individual sub-watersheds havin their own self-contained JSON description -- it's highly readable, and IMO intuitive to describe new things, and extensible by definition.

@timcera
Copy link
Contributor Author

timcera commented Jul 29, 2024

My immediate reaction to JSON is, well, ... let's just say I don't like it.

"These new-fangled phones. How do you type a phone number with one button?"

That being said, maybe it's a good idea. The advantage to CSVs is that you can take advantage of spreadsheets, GIS, Pandas, R (which I admit all might support JSON in some way or another), however you can tap into the familiarity that people have with CSVs. JSON in my mind is similar to XML which is appropriate for some things, however tiresome and tedious to read or edit.

Maybe could support both, though. Just have to come up with the JSON standard. Having all parameters for a particular RCHRES in a single JSON file is possibly interesting, but is a different way of thinking about things compared to a UCI (obviously).

The standard for the imagined CSVs is that they would follow the HSPF tables one-to-one, just comma delimited rather than fixed width. Looking at the link you sent for SEDTRN - it is one HUGE table. I am not imagining anything like that.

@rburghol
Copy link
Contributor

Having had the same reaction to JSON as @timcera initially, I can say happily that while XML has been nothing but a headache to me, JSON is a much more friendly platform. But, to your point, the adoption would require agreeing on an approach. That said, much of the standard guts of JSON just work with numeric and character attributes, so, there is less to do than one might think. And we would be free to have a single RCHRES in a single file, or multiple RCHRESs, separately in a single file, or, even having one block of JSON for one UCI block (though that might undermine the readability benefit of JSON).

The idea on the one CSV to one table is straight-forward of course, I just included that CBP example for reference, although, I think their lesson is that having a single file for every single UCI block can get tedious.

Agree wholeheartedly that this doesn't need to be a one approach only type of endeavor.

@mishranurag
Copy link
Contributor

mishranurag commented Jul 29, 2024 via email

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants