Issues when running FLORIS 4.1.1

[misi9170]

Discussed in #950

^{Originally posted by prith-gs July 26, 2024}
Hello,

I'm trying to run example 01 with FLORIS 4.1.1 and I don't get this error in FLORIS 3.5 and I'm not sure why I get this error.

Here's the code I'm running where I am reading the input yaml file using FlorisModel function in line 9 and I'm using a function I found online in line 4 and 5.

Line 4 and 5 executes properly and has no issues in reading the yaml file and I can see the variable 'data' as well

Here's what's stored in the variable data

But line 9 gives me the following error and I'm not clear why I'm getting this error

Any help is much appreciated.
Thanks!

[misi9170]

Tagging @prith-gs and @HDW7812

[misi9170]

Bringing over comments from discussion:

Comment by @HDW7812:
Hi, I'm also experiencing this problem, although I don't know exactly what's causing it, but I have a feeling it's a conflict with a previously installed package, so the turbine_library_path is incorrect. I create a new envs (without pip install floris), then run the demo, it works. I hope it can help you!
If you are in a hurry to use it, you can try this method. Wish someone could answer you more professionally.

Comment by @misi9170:
Hi @prith-gs ,

If you are using this within a conda environment, could you share the list of packages you have installed? As @HDW7812 pointed out, this could be something to do with package versions in your environment, especially if you are using the same environment you were running v3.5 with.

As also pointed out by @HDW7812 , starting with a fresh conda environment and a fresh pip install of FLORIS may solve your issue.

I'm going to convert this discussion to an issue in the meantime.

Cheers,
Misha

[prith-gs]

Hi @misi9170,

I tried creating a new environment and install floris again and I'm still getting the same error.
I'm using miniconda and here's the list of packages I have installed in the new environment I created.

alabaster                     0.7.12
arrow                         1.2.3
astroid                       2.14.2
asttokens                     2.0.5
atomicwrites                  1.4.0
attrs                         23.1.0
autopep8                      2.0.4
Babel                         2.11.0
backcall                      0.2.0
bcrypt                        3.2.0
beautifulsoup4                4.12.3
binaryornot                   0.4.4
black                         24.4.2
bleach                        4.1.0
Brotli                        1.0.9
certifi                       2024.7.4
cffi                          1.16.0
chardet                       4.0.0
charset-normalizer            3.3.2
click                         8.1.7
cloudpickle                   3.0.0
colorama                      0.4.6
coloredlogs                   15.0.1
comm                          0.2.1
contourpy                     1.1.1
cookiecutter                  2.6.0
cryptography                  42.0.5
cycler                        0.12.1
debugpy                       1.6.7
decorator                     5.1.1
defusedxml                    0.7.1
diff-match-patch              20200713
dill                          0.3.8
docstring-to-markdown         0.11
docutils                      0.18.1
executing                     0.8.3
fastjsonschema                2.16.2
flake8                        7.0.0
FLORIS                        4.1.1
fonttools                     4.53.1
humanfriendly                 10.0
idna                          3.7
imagesize                     1.4.1
importlib-metadata            7.0.1
importlib_resources           6.4.0
inflection                    0.5.1
intervaltree                  3.1.0
ipykernel                     6.28.0
ipython                       8.12.2
isort                         5.13.2
jaraco.classes                3.2.1
jedi                          0.19.1
jellyfish                     1.0.1
Jinja2                        3.1.4
jsonschema                    4.19.2
jsonschema-specifications     2023.7.1
jupyter_client                8.6.0
jupyter_core                  5.7.2
jupyterlab-pygments           0.2.2
keyring                       24.3.1
kiwisolver                    1.4.5
lazy-object-proxy             1.10.0
markdown-it-py                2.2.0
MarkupSafe                    2.1.3
matplotlib                    3.7.5
matplotlib-inline             0.1.6
mccabe                        0.7.0
mdurl                         0.1.0
mistune                       2.0.4
more-itertools                10.1.0
mypy-extensions               1.0.0
nbclient                      0.8.0
nbconvert                     7.10.0
nbformat                      5.9.2
nest-asyncio                  1.6.0
numexpr                       2.8.6
numpy                         1.24.4
numpydoc                      1.5.0
packaging                     24.1
pandas                        2.0.3
pandocfilters                 1.5.0
paramiko                      2.8.1
parso                         0.8.3
pathspec                      0.10.3
pexpect                       4.8.0
pickleshare                   0.7.5
pillow                        10.4.0
pip                           24.0
pkgutil_resolve_name          1.3.10
platformdirs                  3.10.0
pluggy                        1.0.0
ply                           3.11
prompt-toolkit                3.0.43
psutil                        5.9.0
ptyprocess                    0.7.0
pure-eval                     0.2.2
pycodestyle                   2.11.1
pycparser                     2.21
pydocstyle                    6.3.0
pyflakes                      3.2.0
Pygments                      2.15.1
pylint                        2.16.2
pylint-venv                   3.0.3
pyls-spyder                   0.4.0
PyNaCl                        1.5.0
pyparsing                     3.1.2
PyQt5                         5.15.10
PyQt5-sip                     12.13.0
PyQtWebEngine                 5.15.6
pyreadline3                   3.4.1
PySocks                       1.7.1
python-dateutil               2.9.0.post0
python-lsp-black              2.0.0
python-lsp-jsonrpc            1.1.2
python-lsp-server             1.10.0
python-slugify                5.0.2
pytoolconfig                  1.2.6
pytz                          2024.1
pywin32                       305.1
pywin32-ctypes                0.2.2
PyYAML                        6.0.1
pyzmq                         25.1.2
QDarkStyle                    3.2.3
qstylizer                     0.2.2
QtAwesome                     1.2.2
qtconsole                     5.5.1
QtPy                          2.4.1
referencing                   0.30.2
requests                      2.32.3
rich                          13.7.1
rope                          1.12.0
rpds-py                       0.10.6
Rtree                         1.0.1
scipy                         1.10.1
setuptools                    69.5.1
shapely                       2.0.5
sip                           6.7.12
six                           1.16.0
snowballstemmer               2.2.0
sortedcontainers              2.4.0
soupsieve                     2.5
Sphinx                        5.0.2
sphinxcontrib-applehelp       1.0.2
sphinxcontrib-devhelp         1.0.2
sphinxcontrib-htmlhelp        2.0.0
sphinxcontrib-jsmath          1.0.1
sphinxcontrib-qthelp          1.0.3
sphinxcontrib-serializinghtml 1.1.5
spyder                        5.5.1
spyder-kernels                2.5.0
stack-data                    0.2.0
text-unidecode                1.3
textdistance                  4.2.1
three-merge                   0.1.1
tinycss2                      1.2.1
tomli                         2.0.1
tomlkit                       0.11.1
tornado                       6.4.1
traitlets                     5.14.3
typing_extensions             4.11.0
tzdata                        2024.1
ujson                         5.10.0
Unidecode                     1.2.0
urllib3                       2.2.2
watchdog                      4.0.1
wcwidth                       0.2.5
webencodings                  0.5.1
whatthepatch                  1.0.2
wheel                         0.43.0
win-inet-pton                 1.1.0
wrapt                         1.14.1
yapf                          0.40.2
zipp                          3.17.0

please do let me know if you need any other information. Thanks!

[Bartdoekemeijer]

Hi @prith-gs,

What is your turbine_library_path?

[misi9170]

@jfrederik-nrel , @Bartdoekemeijer 's comment above reminded me---was this the same issue I talked about with you a few months back? I recall an issue with loading turbines having to do with a piece of code not correctly building a path when running on Windows

[jfrederik-nrel]

I don't think it's the same thing to be honest, I think we resolved that issue. For reference, the issue you are referring to is #873.

[misi9170]

I don't think it's the same thing to be honest, I think we resolved that issue. For reference, the issue you are referring to is #873.

Ah ok, thanks @jfrederik-nrel !

[paulf81]

I added windows back to continuous-integration and there is an issue with Windows for python 3.8 and 3.9,

PR #953

[misi9170]

This does seem to be the issue encountered by @prith-gs , screenshot from one of the failed tests above:

That seems to confirm it's an error specific to Windows OS. We will look into a fix over on #953

[prith-gs]

Hello @misi9170 @paulf81 @Bartdoekemeijer

I was indeed using python v3.8.1 and had this issue for FLORIS v4.x and not for v3.x.
Now, when I switched to python v3.11.9, I don't get that error anymore and the example files work properly.

Appreciate the help! :)

Will this be sorted for python v3.8 and 3.9 in further releases? or do I have to switch to python >= v3.10 for FLORIS v4.0 and above?

Thanks!

[misi9170]

@prith-gs ,

Glad that you got this working. Right, the issue here seems to be limited to older python versions (3.8 and 3.9, and likely earlier versions) running on Windows systems. We will likely not try to address this, and shift support to python 3.10 and above. I will discuss with other Floris developers and confirm.

At this stage, do you have a specific need to run Floris with an earlier version of python?

Cheers,
Misha

[prith-gs]

@misi9170

I'm working with some inhouse packages which are written for python 3.8 and hence I'm using FLORIS with an older version of python. If it can be modified to work with the previous version that'll be nice, if not it's still fine, we are probably due for an upgrade anyway. Appreciate the help!

BR,
Pritheesh

[Bartdoekemeijer]

Yet another reason to upgrade to Python 3.11 or higher: https://docs.python.org/3/whatsnew/3.11.html.

Python 3.11 is between 10-60% faster than Python 3.10. On average, we measured a 1.25x speedup on the standard benchmark suite.

[gogannes]

Hey, there.

Thanks for posting this issue! I run into the same issue using python 3.9.
I was first confused as I didn't expect any issue because setup.py specifies the required python version with >=3.8.0.

Maybe one could update setup.py to take the higher minimum python version for windows into account.
The following snippet could help:

import sys
if sys.platform == "win32":
    REQUIRES_PYTHON = ">=3.10.0"  # Minimum Python version for Windows
else:
    REQUIRES_PYTHON = ">=3.8.0"  # Minimum Python version for other platforms

Best,
Johannes

[misi9170]

Thanks for the suggestion, Johannes!

Agreed that the versioning listed in setup.py is misleading, and thank you for the suggestion of having platform-depending versions. We've actually recently been discussing moving to relying solely on pyproject.toml, rather than setup.py, for package requirements. At that stage, we may shift to requiring python >= 3.10 across all operating systems, as 3.8 just recently reached end-of-life and 3.9 will be at end-of-life in a year's time.

We will try to post updates in this issue thread as we make decisions regarding package management updates and python version support.

Cheers,
Misha

[rafmudaf]

I spent some time looking into this and here's what I've found. The issue in FLORIS is located in type_dec.convert_to_path. This function is meant to resolve a truncated path to either an absolute path (i.e. starting with /Users/... for macOS or C:\Users\... for Windows) or relative path from the Python interpreter's current working directory. The absolute path part works correctly, but the relative path search has a subtle issue (maybe a bug, maybe not, and maybe it's not our fault).

To find the relative path, the function uses the inspect.stack module to get the filename of the file that called the current function, and then it will build a relative path from there. The frames in the call stack have the following structure (the details are redacted so you can see the structure):

FrameInfo(
    frame=<frame at ..., file 'file path...', line ..., code ...>,
    filename='C:\\hostedtoolcache\\...',
    lineno=103,
    function='_multicall',
    code_context=['                    res = hook_impl.function(*args)\n'],
    index=0
),

However, since FLORIS uses attrs, the "filename" for attrs-derived classes is instead a function: <attrs generated init floris.core.farm.Farm>. Since all classes in floris.core are attrs-derived, I think using inspect.stack()[1] to create a path will not work. There is a unit test for this, and it passes because it is not testing with an attrs-derived class. That is to say that the code works well, but it is not applicable to the structure of FLORIS classes.

#951 (comment) points out that this works for Python 3.10 and above, so why is that and how does this work at all? The different behavior across Python versions is due to a change in pathlib and os.path that introduced the concept of "strict" path resolutions and consolidated the path resolving. In Python 3.9, pathlib.Path has a Windows-specific Path class and a unique resolve() method. Despite accepting the "strict" argument, the implementation is incomplete because it still checks for valid characters in the path. In Python 3.10, the Path classes for all system types were consolidated and they ultimately use os.path.realpath for Path.resolve. Also in Python 3.10, os.path.realpath got the "strict" parameter and it was defaulted to False. Basically, in Python 3.10+, an invalid path in Path.resolve() is treated as if it might exist regardless of the content of the path string, whereas in lower Pythons the path string is checked for at least having valid characters.

For what it's worth, there have been multiple suggestions to change this behavior in pathlib for Python <3.10:

• https://bugs.python.org/issue35306
• pathlib.exists on Windows raises an exception on URL like/bad input python/cpython#87021 (comment)
• [Windows] OSError when testing whether pathlib.Path('*') exists python/cpython#79487 (comment)

A similar backwards compatible change was made for pathlib.Path.exists in Python 3.8 and Python 3.9, but it did not include the change for .resolve. I plan to propose the change to the CPython maintainers, but I also assume that is a stretch and a very long path.

As for FLORIS, I think the fix is to avoid the relative path search from the file of the calling-function. Currently, this will always be an attrs function and it was only "working" due to a nuance in error handling from the Python pathlib library. I've shown this in https://github.com/NREL/floris/pull/953/files. @RHammond2 @bayc your opinions here would be very valuable (plus @RHammond2 I thought you might enjoy the Python deep dive, as well 😺) since you are more aware of the consequences of how various files are loaded.

For what it's worth, I also made a simple repository to show that this is indeed an issue across Python versions for Windows: https://github.com/rafmudaf/windows_path_error/actions/runs/11899793288.

[RHammond2]

Thanks for sharing this comprehensive writeup, @paulf81 ! For what it's worth Python 3.9 is no longer actively supported, and is ending security updates as of October 31, 2025. Moreover, Python 3.8 is officially unsupported, so my thought would be to end support for Python 3.8/9 given it's a 3.8/9 issue. In a couple of other projects where we were upgrading infrastructure, we also made the decision to support 3.10+ in their latest releases.

As for how to handle the attrs derived relative paths correctly, I don't have any immediate resolutions, but I suspect HOPP/GreenHEART will also encounter this issue because they use a direct copy of this implementation, and so I can share back any solution we create.

[paulf81]

Thanks @RHammond2 for your helpful comments!