pytest failures: OSErrors and pyomo ApplicationErrors on M3

[dakotaramos]

Summary

Creating issue per discussions in code coordination meeting.

Multiple users on M3 are seeing pytest failures, OSErrors around architecture x86_64 vs arm64e or arm64, and pyomo ApplicationErrors around abnormal exit of the cbc solver.

Full CLI output with errors found in attached file. Followed the HOPP install instructions as detailed in the ReadME.md. Confirmed bash is able to use Rosetta to launch an x86_64 shell via arch -x86_64 bash.

greensteel_pytest_results.txt

How to reproduce

clone, create conda env, install:

• cd to folder where you want to clone /HOPP/
• git clone -b greensteel-eco-sync https://github.com/NREL/HOPP.git
• cd HOPP
• conda create --name greensteel_hopp python=3.8 -y
• conda activate greensteel_hopp
• conda install -c conda-forge coin-or-cbc=2.10.8 -y
• conda install -c conda-forge glpk -y
• pip install -r requirements.txt
• pip install -r requirements-dev.txt
• pip install -e .

Set NREL_API_KEY and NREL_API_EMAIL as env variables

test install:

• cd to HOPP/ directory
• pytest

Relevant output

See attached greensteel_pytest_results.txt for full CLI output of pytest and errors

Example OSError:
OSError: dlopen(/Users/dpotere/code_dev/Git/greensteel_hopp/HOPP/hopp/simulation/technologies/csp/pySSC_daotk/libs/ssc.dylib, 0x0006): tried: '/Users/dpotere/code_dev/Git/greensteel_hopp/HOPP/hopp/simulation/technologies/csp/pySSC_daotk/libs/ssc.dylib' (mach-o file, but is an incompatible architecture (have 'x86_64', need 'arm64e' or 'arm64')), '/System/Volumes/Preboot/Cryptexes/OS/Users/dpotere/code_dev/Git/greensteel_hopp/HOPP/hopp/simulation/technologies/csp/pySSC_daotk/libs/ssc.dylib' (no such file), '/Users/dpotere/code_dev/Git/greensteel_hopp/HOPP/hopp/simulation/technologies/csp/pySSC_daotk/libs/ssc.dylib' (mach-o file, but is an incompatible architecture (have 'x86_64', need 'arm64e' or 'arm64'))

/opt/miniconda3/envs/greensteel_hopp/lib/python3.8/ctypes/init.py:373: OSError

Example pyomo ApplicationError:

pyomo.common.errors.ApplicationError: Solver (cbc) did not exit normally

/opt/miniconda3/envs/greensteel_hopp/lib/python3.8/site-packages/pyomo/opt/base/solvers.py:628: ApplicationError

HOPP version

2.2.0

System Information

• OS: macOS 14.5
• Python version: 3.8.19
• Library versions
  about-time==4.2.1
  accumulation_tree==0.6.2
  aiounittest==1.4.2
  alabaster==0.7.13
  alembic==1.13.1
  alive-progress==3.1.5
  anyio==4.4.0
  appnope==0.1.4
  appscript==1.2.5
  argon2-cffi==23.1.0
  argon2-cffi-bindings==21.2.0
  arrow==1.3.0
  asttokens==2.4.1
  async-lru==2.0.4
  attrs==23.2.0
  autograd==1.6.2
  Babel==2.15.0
  backcall==0.2.0
  backoff==2.2.1
  bayesian-optimization==1.4.3
  beautifulsoup4==4.12.3
  bleach==6.1.0
  bokeh==3.1.1
  certifi==2024.6.2
  cffi==1.16.0
  charset-normalizer==3.3.2
  click==8.1.7
  cma==3.2.2
  colorama==0.4.6
  coloredlogs==15.0.1
  colorlog==6.8.2
  comm==0.2.2
  contexttimer==0.3.3
  contourpy==1.1.1
  CoolProp==6.6.0
  coverage==7.5.3
  cycler==0.12.1
  Cython==3.0.10
  debugpy==1.8.1
  decorator==5.1.1
  deepdiff==7.0.1
  defusedxml==0.7.1
  Deprecated==1.2.14
  dill==0.3.8
  diskcache==5.6.3
  docopt==0.6.2
  docutils==0.18.1
  Electrolyzer @ git+https://github.com/jaredthomas68/electrolyzer.git@f113f9596f508fddd7fef42ffc2d9ee433f0e8d5
  entrypoints==0.4
  et-xmlfile==1.1.0
  exceptiongroup==1.2.1
  executing==2.0.1
  fastjsonschema==2.20.0
  fastkml==0.12
  flatten-dict==0.4.2
  FLORIS==3.5
  fonttools==4.53.0
  fqdn==1.5.1
  funcy==2.0
  future==1.0.0
  getjson==2.0.0
  global-land-mask==1.0.0
  grapheme==0.6.0
  greenlet==3.0.3
  h11==0.14.0
  h3==3.7.7
  -e git+https://github.com/NREL/HOPP.git@37d4fb7f16657426edd87442359226a4281af7d4#egg=HOPP
  httpcore==1.0.5
  httpx==0.27.0
  humanfriendly==10.0
  humpday==0.7.1
  hybridbosse==0.9.6
  idna==3.7
  imagesize==1.4.1
  importlib_metadata==7.1.0
  importlib_resources==6.4.0
  iniconfig==2.0.0
  ipykernel==6.29.4
  ipympl==0.9.3
  ipyparallel==8.8.0
  ipython==8.12.3
  ipython-genutils==0.2.0
  ipywidgets==8.1.3
  isoduration==20.11.0
  jax==0.4.13
  jaxlib==0.4.13
  jedi==0.19.1
  Jinja2==3.1.4
  joblib==1.4.2
  json5==0.9.25
  jsonpointer==3.0.0
  jsonschema==4.22.0
  jsonschema-specifications==2023.12.1
  jupyter-book==0.14.0
  jupyter-cache==0.6.1
  jupyter-events==0.10.0
  jupyter-lsp==2.2.5
  jupyter_client==8.6.2
  jupyter_core==5.7.2
  jupyter_server==2.14.1
  jupyter_server_terminals==0.5.3
  jupyterlab==4.2.2
  jupyterlab_pygments==0.3.0
  jupyterlab_server==2.27.2
  jupyterlab_widgets==3.0.11
  kiwisolver==1.4.5
  landscapes==0.0.12
  latexcodec==3.0.0
  lcoe==0.3.2
  linkify-it-py==2.0.3
  lxml==5.2.2
  Mako==1.3.5
  markdown-it-py==2.2.0
  MarkupSafe==2.1.5
  marmot-agents==0.2.5
  matplotlib==3.7.5
  matplotlib-inline==0.1.7
  mdit-py-plugins==0.3.5
  mdurl==0.1.2
  microconventions==1.0.0
  mistune==3.0.2
  ml-dtypes==0.2.0
  muid==0.5.3
  multiprocessing_on_dill==3.5.0a4
  myst-nb==0.17.2
  myst-parser==0.18.1
  nbclient==0.7.4
  nbconvert==7.16.4
  nbformat==5.10.4
  nest-asyncio==1.6.0
  networkx==3.1
  nevergrad==1.0.2
  notebook==7.2.1
  notebook_shim==0.2.4
  NREL-PySAM==4.2.0
  num2words==0.5.13
  numexpr==2.8.6
  numpy==1.24.4
  numpy-financial==1.0.0
  numpydoc==1.6.0
  openmdao==3.33.0
  openpyxl==3.1.4
  opt-einsum==3.3.0
  optuna==3.6.1
  orbit-nrel @ git+https://github.com/WISDEM/ORBIT.git@3baed36052d7503de6ea6f2db0b40945d637ea35
  ordered-set==4.1.0
  overrides==7.7.0
  packaging==24.1
  pandas==2.0.3
  pandocfilters==1.5.1
  parameterized==0.9.0
  parso==0.8.4
  pathlib==1.0.1
  patsy==0.5.6
  pexpect==4.9.0
  pickleshare==0.7.5
  pillow==10.3.0
  Pint==0.21.1
  pkgutil_resolve_name==1.3.10
  platformdirs==4.2.2
  playwright==1.44.0
  plotly==5.22.0
  pluggy==1.5.0
  ply==3.11
  POAP==0.1.26
  ProFAST @ git+https://github.com/NREL/ProFAST.git@9751d1f9909767c4d55a8b4d183413574f3043d3
  prometheus_client==0.20.0
  prompt_toolkit==3.0.47
  psutil==5.9.8
  ptyprocess==0.7.0
  pure-eval==0.2.2
  pvmismatch==4.1
  pyaml==24.4.0
  pyarrow==16.1.0
  pybtex==0.24.0
  pybtex-docutils==1.0.3
  pycodestyle==2.12.0
  pycparser==2.22
  pydata-sphinx-theme==0.12.0
  pydocstyle==2.0.0
  pyDOE2==1.3.0
  pyDOE3==1.0.2
  pyee==11.1.0
  pygeoif==0.7
  Pygments==2.18.0
  pymoo==0.6.1.1
  pymorton==1.0.5
  Pyomo==6.7.3
  pyparsing==3.1.2
  pyproj==3.5.0
  pysolar==0.11
  pySOT==0.3.3
  pytest==8.2.2
  pytest-subtests==0.12.1
  python-benedict==0.33.2
  python-dateutil==2.9.0.post0
  python-dotenv==1.0.1
  python-fsutil==0.14.1
  python-json-logger==2.0.7
  python-rapidjson==1.17
  python-slugify==8.0.4
  pytz==2024.1
  pyudorandom==1.0.0
  PyYAML==6.0.1
  pyyaml-include==1.4.1
  pyzmq==26.0.3
  rainflow==3.2.0
  referencing==0.35.1
  requests==2.32.3
  responses==0.25.3
  rfc3339-validator==0.1.4
  rfc3986-validator==0.1.1
  rpds-py==0.18.1
  ruamel.yaml==0.18.6
  ruamel.yaml.clib==0.2.8
  schema==0.7.7
  scikit-learn==1.3.2
  scikit-optimize==0.10.2
  scipy==1.10.1
  Send2Trash==1.8.3
  Shapely==1.8.5.post1
  simpy==4.1.1
  six==1.16.0
  sniffio==1.3.1
  snowballstemmer==2.2.0
  soupsieve==2.5
  Sphinx==5.0.2
  sphinx-book-theme==0.4.0rc1
  sphinx-comments==0.0.3
  sphinx-copybutton==0.5.2
  sphinx-jupyterbook-latex==0.5.2
  sphinx-multitoc-numbering==0.1.3
  sphinx-rtd-theme==2.0.0
  sphinx-sitemap==2.6.0
  sphinx-thebe==0.2.1
  sphinx-togglebutton==0.3.2
  sphinx_design==0.3.0
  sphinx_external_toc==0.3.1
  sphinxcontrib-applehelp==1.0.4
  sphinxcontrib-bibtex==2.5.0
  sphinxcontrib-devhelp==1.0.2
  sphinxcontrib-htmlhelp==2.0.1
  sphinxcontrib-jquery==4.1
  sphinxcontrib-jsmath==1.0.1
  sphinxcontrib-qthelp==1.0.3
  sphinxcontrib-serializinghtml==1.1.5
  SQLAlchemy==2.0.30
  stack-data==0.6.3
  statsmodels==0.14.1
  tabulate==0.9.0
  tdigest==0.5.2.2
  tenacity==8.4.1
  terminado==0.18.1
  testflo==1.4.18
  text-unidecode==1.3
  threadpoolctl==3.5.0
  timezonefinder==6.5.2
  tinycss2==1.3.0
  tomli==2.0.1
  tornado==6.4.1
  tqdm==4.66.4
  traitlets==5.14.3
  types-python-dateutil==2.9.0.20240316
  typing_extensions==4.12.2
  tzdata==2024.1
  uc-micro-py==1.0.3
  uri-template==1.3.0
  urllib3==2.2.2
  utm==0.7.0
  wcwidth==0.2.13
  webcolors==24.6.0
  webencodings==0.5.1
  websocket-client==1.8.0
  websockets==12.0
  widgetsnbextension==4.0.11
  winning==1.0.3
  wrapt==1.16.0
  xlwings==0.31.5
  xyzservices==2024.6.0
  zipp==3.19.2