Skip to content

Commit

Permalink
Merge pull request #476 from NREL/release_v2024.11.0
Browse files Browse the repository at this point in the history
  • Loading branch information
rajeee authored Nov 15, 2024
2 parents 4adb9f9 + a807bc8 commit a15986c
Show file tree
Hide file tree
Showing 6 changed files with 201 additions and 135 deletions.
8 changes: 4 additions & 4 deletions buildstockbatch/__version__.py
Original file line number Diff line number Diff line change
Expand Up @@ -4,9 +4,9 @@
__title__ = "buildstockbatch"
__description__ = "Executing BuildStock projects on batch infrastructure."
__url__ = "http://github.com/NREL/buildstockbatch"
__version__ = "2023.11.0"
__schema_version__ = "0.3"
__author__ = "Noel Merket"
__author_email__ = "[email protected]"
__version__ = "2024.11.0"
__schema_version__ = "0.4"
__author__ = "Noel Merket, Rajendra Adhikari"
__author_email__ = "[email protected], [email protected]"
__license__ = "BSD-3"
__copyright__ = "Copyright {} The Alliance for Sustainable Energy".format(dt.date.today().year)
129 changes: 129 additions & 0 deletions docs/changelog/changelog_2024_11_0.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,129 @@
====================
v2024.11.0 Changelog
====================

.. changelog::
:version: v2024.11.0
:released: 2024-11-05


.. change::
:tags: general
:pullreq: 421

Refactor docker_base to use inversion of control so that it can more strongly and easily ensure consistency
between various implementations (GCP implementation to come). This also includes teasing apart the several batch
prep steps (weather, assets, and jobs) into their own methods so they can each be more easily understood,
shared, and maintained.

.. change::
:tags: general
:pullreq: 422

Refactor AWS code so it can be shared by the upcoming GCP implementation.

.. change::
:tags: general, bugfix
:pullreq: 426

A bugfix for gracefully handling empty data_point_out.json files.

.. change::
:tags: aws, feature
:pullreq: 345

Major update to get AWS Batch run environment working.

.. change::
:tags: general
:pullreq: 435

Add helper to log a summary of how many simulations succeeded and failed at the end of a job.

.. change::
:tags: general, feature
:pullreq: 437

Add a ``step_failures`` section to json results files with error messages from OpenStudio simulations.

.. change::
:tags: general
:pullreq: 436

Clean up handling of weather files in GCP/AWS implementations: only upload files that are required,
and fail with clearer messaging if any files are missing.

.. change::
:tags: general, feature
:pullreq: 423

Add GCP implementation.

.. change::
:tags: bugfix, schema
:pullreq: 450

Requires ``os_version`` and ``os_sha`` in the project file.

.. change::
:tags: general
:pullreq: 456

Refactor WorkflowGenerator.

.. change::
:tags: general, feature
:pullreq: 458

Add a new version (2024.07.19) of the Residential HPXML Workflow Generator that
changes UpgradeCosts from reporting measure to a regular measure. ReportHPXMLOutput
is no longer called. This feature is created to support the corresponding update in
ResStock (https://github.com/NREL/resstock/pull/1253)

To facilitate the creation of new version, the workflow generator code base is refactored
to have one folder for each version. New yaml schema (v0.4) is created that defines a
`version` key for the `workflow_generator` block. The base class loads the appropriate
version of the workflow_generator based on the version key. If version key is missing
(when using older schema), it is defaulted to the oldest available version (2024.07.18).

.. change::
:tags: general, feature
:pullreq: 461

Add a new version (2024.07.20) of the Residential HPXML Workflow Generator that
exposes optional ``include_annual_bills`` (defaults to true) and
``include_monthly_bills`` (defaults to false) arguments for reporting annual
and monthly utility bill outputs, respectively.

.. change::
:tags: general, bugfix
:pullreq: 464

Stop creating dask _metadata files for the timeseries parquet files since it crashes the
postprocessing.

.. change::
:tags: bugfix, hpc
:pullreq: 467

Updates the bind mount for apptainer to include the whole resources
directory instead of just the hpxml-measures directory. Makes it work
with newer versions of ResStock.

.. change::
:tags: general
:pullreq: 472

Remove support for Python 3.8, 3.9, and 3.10. Add support for Python 3.12.

.. change::
:tags: general
:pullreq: 472

Remove eagle related code and references since eagle is retired.

.. change::
:tags: general
:pullreq: 465

Append schedules to the timeseries files.
125 changes: 0 additions & 125 deletions docs/changelog/changelog_dev.rst
Original file line number Diff line number Diff line change
Expand Up @@ -14,128 +14,3 @@ Development Changelog
This is an example change. Please copy and paste it - for valid tags please refer to ``conf.py`` in the docs
directory. ``pullreq`` should be set to the appropriate pull request number and ``tickets`` to any related
github issues. These will be automatically linked in the documentation.

.. change::
:tags: general
:pullreq: 421

Refactor docker_base to use inversion of control so that it can more strongly and easily ensure consistency
between various implementations (GCP implementation to come). This also includes teasing apart the several batch
prep steps (weather, assets, and jobs) into their own methods so they can each be more easily understood,
shared, and maintained.

.. change::
:tags: general
:pullreq: 422

Refactor AWS code so it can be shared by the upcoming GCP implementation.

.. change::
:tags: general, bugfix
:pullreq: 426

A bugfix for gracefully handling empty data_point_out.json files.

.. change::
:tags: aws, feature
:pullreq: 345

Major update to get AWS Batch run environment working.

.. change::
:tags: general
:pullreq: 435

Add helper to log a summary of how many simulations succeeded and failed at the end of a job.

.. change::
:tags: general, feature
:pullreq: 437

Add a ``step_failures`` section to json results files with error messages from OpenStudio simulations.

.. change::
:tags: general
:pullreq: 436

Clean up handling of weather files in GCP/AWS implementations: only upload files that are required,
and fail with clearer messaging if any files are missing.

.. change::
:tags: general, feature
:pullreq: 423

Add GCP implementation.

.. change::
:tags: bugfix, schema
:pullreq: 450

Requires ``os_version`` and ``os_sha`` in the project file.

.. change::
:tags: general
:pullreq: 456

Refactor WorkflowGenerator.

.. change::
:tags: general, feature
:pullreq: 458

Add a new version (2024.07.19) of the Residential HPXML Workflow Generator that
changes UpgradeCosts from reporting measure to a regular measure. ReportHPXMLOutput
is no longer called. This feature is created to support the corresponding update in
ResStock (https://github.com/NREL/resstock/pull/1253)

To facilitate the creation of new version, the workflow generator code base is refactored
to have one folder for each version. New yaml schema (v0.4) is created that defines a
`version` key for the `workflow_generator` block. The base class loads the appropriate
version of the workflow_generator based on the version key. If version key is missing
(when using older schema), it is defaulted to the oldest available version (2024.07.18).

.. change::
:tags: general, feature
:pullreq: 461

Add a new version (2024.07.20) of the Residential HPXML Workflow Generator that
exposes optional ``include_annual_bills`` (defaults to true) and
``include_monthly_bills`` (defaults to false) arguments for reporting annual
and monthly utility bill outputs, respectively.

.. change::
:tags: general, bugfix
:pullreq: 464

Stop creating dask _metadata files for the timeseries parquet files since it crashes the
postprocessing.

.. change::
:tags: general, bugfix
:pullreq: 466

Add a flag to allow only uploading the missing files to s3.
:tags: bugfix, hpc
:pullreq: 467

Updates the bind mount for apptainer to include the whole resources
directory instead of just the hpxml-measures directory. Makes it work
with newer versions of ResStock.

.. change::
:tags: general
:pullreq: 472

Remove support for Python 3.8, 3.9, and 3.10. Add support for Python 3.12.

.. change::
:tags: general
:pullreq: 472

Remove eagle related code and references since eagle is retired.

.. change::
:tags: general
:pullreq: 465

Append schedules to the timeseries files.
2 changes: 2 additions & 0 deletions docs/changelog/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,7 @@ Current Migration Guide
.. toctree::
:titlesonly:

migration_2024_11_0
migration_2023_11_0

Changelogs
Expand All @@ -19,6 +20,7 @@ Changelogs
.. toctree::
:titlesonly:

changelog_2024_11_0
changelog_2023_11_0
changelog_2023_10_0
changelog_2023_06_0
Expand Down
54 changes: 54 additions & 0 deletions docs/changelog/migration_2024_11_0.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,54 @@
.. |version| replace:: v2024.11.0

=======================================
What's new in buildstockbatch |version|
=======================================

.. admonition:: About this Document

This document describes changes between buildstockbatch version 2023.11.0 and
buildstockbatch version |version|

Major Changes
=============

Below are the major changes in this release.

Eagle is retired
----------------

This version should be backwards compatible with previous versions of
buildstockbatch except for being able to run in Eagle. Since Eagle is retired, BuildStockBatch no
longer supports buildstock_eagle command. If you are just comming upto speed with Kestrel, you can
use your existing yaml file by replacing 'eagle' with 'kestrel' and using buildstock_kestrel
command instead of buildstock_eagle command. See how to run in Kestrel :ref:`kestrel-run`.

BuildStockBatch uses Python 3.11
--------------------------------

BuildStockBatch now uses Python 3.11. If you are using a version of Python
older than 3.11 in your local buildstockbatch environment, you will need to upgrade the python
version or create a new environment with python 3.11 (or 3.12).

Timeseries files include schedules
----------------------------------

In ResStock runs, the schedules are now appended to the timeseries files.

WorkflowGenerator is versioned
------------------------------

`The workflow generator needs a version specification. <https://github.com/NREL/resstock/blob/v3.3.0/project_national/national_baseline.yml#L17>`_
Older yaml files without version specification will default to version 2024.07.18 (the assigned version before versioning was introduced).
If you are using a custom branch of ResStock, you should use the same version of the workflow generator as that in the ResStock branch
it is based on.

Google Cloud Platform (GCP) support
-----------------------------------

BuildStockBatch now supports GCP. See :ref:`gcp-config` and :ref:`gcp-run`.


See :doc:`changelog_2024_11_0` for full details of what changed.


18 changes: 12 additions & 6 deletions docs/run_sims.rst
Original file line number Diff line number Diff line change
Expand Up @@ -20,8 +20,10 @@ Running a project file is straightforward. Call the ``buildstock_local`` command
Running the simulation with ``--postprocessonly`` when there is already postprocessed results from previous run will
overwrite those results.

NREL HPC (Kestrel)
~~~~~~~~~~~~~~~~~~~
.. _kestrel-run:

Running on NREL HPC (Kestrel)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
After you have :ref:`activated the appropriate conda environment on Kestrel <kestrel_install>`,
you can submit a project file to be simulated by passing it to the ``buildstock_kestrel`` command.

Expand Down Expand Up @@ -72,8 +74,10 @@ fail. If most of the jobs succeeded, rather than rerun everything you can
resubmit just the jobs that failed with the ``--rerun_failed`` command line
argument. This will also clear out and rerun the postprocessing.

Amazon Web Services
~~~~~~~~~~~~~~~~~~~
.. _aws-run:

Running on Amazon Web Services
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Running a batch on AWS is done by calling the ``buildstock_aws`` command line
tool.
Expand Down Expand Up @@ -124,8 +128,10 @@ This will clean up all the AWS resources that were created on your behalf to run
the simulations. Your results will still be on S3 and queryable in Athena.


Google Cloud Platform
~~~~~~~~~~~~~~~~~~~~~
.. _gcp-run:

Running on Google Cloud Platform
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Run a project on GCP by calling the ``buildstock_gcp`` command line tool.

Expand Down

0 comments on commit a15986c

Please sign in to comment.