diff --git a/README.md b/README.md index 2a8602c1d..0257434a4 100644 --- a/README.md +++ b/README.md @@ -80,7 +80,7 @@ Example RESTful interaction: | Receive current test scenario. | GET ``scenario`` | | Set test scenario. Setting the argument ``time_period`` performs an initialization with predefined start time and warmup period and will only simulate for predefined duration. | PUT ``scenario`` with optional arguments ``electricity_price=``, ``time_period=``. See README in [/testcases](https://github.com/ibpsa/project1-boptest/tree/master/testcases) for options and test case documentation for details.| | Receive BOPTEST version. | GET ``version`` | -| Submit KPIs, other test information, and optional string tags (up to 10) to online dashboard. Requires a formal test scenario to be completed, initialized using the PUT ``scenario`` API. | POST ``submit`` with required argument ``api_key=`` and optional arguments ``tag#=`` where # is an integer between 1 and 10. The API key can be obtained from the user account registered wth the online dashboard at (url coming soon).| +| Submit KPIs, other test information, and optional string tags (up to 10) to online dashboard. Requires a formal test scenario to be completed, initialized using the PUT ``scenario`` API. | POST ``submit`` with required argument ``api_key=`` and optional arguments ``tag#=`` where # is an integer between 1 and 10. The API key can be obtained from the user account registered with the online dashboard.| ## Development @@ -105,7 +105,6 @@ An OpenAI-Gym environment for BOPTEST is implemented in [ibpsa/project1-boptest- A proposed BOPTEST home page and dashboard for creating accounts and sharing results is published here https://xd.adobe.com/view/0e0c63d4-3916-40a9-5e5c-cc03f853f40a-783d/. ## Publications -### To cite, please use: D. Blum, J. Arroyo, S. Huang, J. Drgona, F. Jorissen, H.T. Walnum, Y. Chen, K. Benne, D. Vrabie, M. Wetter, and L. Helsen. (2021). ["Building optimization testing framework (BOPTEST) for simulation-based benchmarking of control strategies in buildings."](https://doi.org/10.1080/19401493.2021.1986574) *Journal of Building Performance Simulation*, 14(5), 586-610. ### Additional publications: @@ -128,3 +127,5 @@ J. Arroyo, F. Spiessens, and L. Helsen. (2020). [“Identification of Multi-zone D. Blum, F. Jorissen, S. Huang, Y. Chen, J. Arroyo, K. Benne, Y. Li, V. Gavan, L. Rivalin, L. Helsen, D. Vrabie, M. Wetter, and M. Sofos. (2019). [“Prototyping the BOPTEST framework for simulation-based testing of advanced control strategies in buildings.”](http://www.ibpsa.org/proceedings/BS2019/BS2019_211276.pdf) In *Proceedings of the 16th International Conference of IBPSA*, Sep 2 – 4. Rome, Italy. S. Huang, Y. Chen, P. W. Ehrlich, and D. L. Vrabie. (2018). [“A Control-Oriented Building Envelope and HVAC System Simulation Model for a Typical Large Office Building.”](https://www.ashrae.org/File%20Library/Conferences/Specialty%20Conferences/2018%20Building%20Performance%20Analysis%20Conference%20and%20SimBuild/Papers/C101.pdf) In *Proceedings of 2018 Building Performance Modeling Conference and SimBuild co-organized by ASHRAE and IBPSA-USA*, Sep 26 - 28. Chicago, IL. + +### To cite, please use: diff --git a/contributors.md b/contributors.md index 19d98c54f..09ce8ee96 100644 --- a/contributors.md +++ b/contributors.md @@ -15,6 +15,7 @@ Thank you to all who have provided guidance on the development of this software. - Donghun Kim, Lawrence Berkeley National Laboratory - Yanfei Li, National Renewable Energy Laboratory - Nicholas Long, National Renewable Energy Laboratory +- Robert Lutes, Pacific Northwest National Laboratory - Christian Veje, University of Southern Denmark - Draguna Vrabie, Pacific Northwest National Laboratory - Harald Walnum, SINTEF diff --git a/releasenotes.md b/releasenotes.md index f2265b720..3ff0964bb 100644 --- a/releasenotes.md +++ b/releasenotes.md @@ -1,8 +1,8 @@ # Release Notes -## BOPTEST v0.x.x +## BOPTEST v0.3.0 -Released on xx/xx/xxxx. +Released on 07/27/2022. **The following changes are backwards-compatible and do not significantly change benchmark results:** diff --git a/testing/references/bestest_air/submit.json b/testing/references/bestest_air/submit.json index a17986bf9..cf87aa497 100644 --- a/testing/references/bestest_air/submit.json +++ b/testing/references/bestest_air/submit.json @@ -1 +1 @@ -{"dash_url": "https://api.boptest.net:8081//api/results", "payload": {"results": [{"account": {"apiKey": "valid_api_key"}, "boptestVersion": "0.x.x\n", "buildingType": {"uid": "bestest_air"}, "controlStep": "86400.0", "dateRun": "2020-05-17 00:00:00", "forecastParameters": {"horizon": 3600.0, "interval": 300.0}, "isShared": true, "kpis": {"cost_tot": 0.0273067557575723, "emis_tot": 0.785588519629367, "ener_tot": 3.722732131848899, "idis_tot": 1220.1797359785448, "pdih_tot": null, "pele_tot": 0.010215810323849649, "pgas_tot": 0.12133181119488147, "tdis_tot": 5.687315396946064, "time_rat": 0}, "scenario": {"electricityPrice": "dynamic", "timePeriod": "peak_heat_day", "weatherForecastUncertainty": "deterministic"}, "tags": ["baseline", "unit_test"], "uid": "1"}]}} +{"dash_url": "https://api.boptest.net:8081//api/results", "payload": {"results": [{"account": {"apiKey": "valid_api_key"}, "boptestVersion": "0.3.0\n", "buildingType": {"uid": "bestest_air"}, "controlStep": "86400.0", "dateRun": "2020-05-17 00:00:00", "forecastParameters": {"horizon": 3600.0, "interval": 300.0}, "isShared": true, "kpis": {"cost_tot": 0.0273067557575723, "emis_tot": 0.785588519629367, "ener_tot": 3.722732131848899, "idis_tot": 1220.1797359785448, "pdih_tot": null, "pele_tot": 0.010215810323849649, "pgas_tot": 0.12133181119488147, "tdis_tot": 5.687315396946064, "time_rat": 0}, "scenario": {"electricityPrice": "dynamic", "timePeriod": "peak_heat_day", "weatherForecastUncertainty": "deterministic"}, "tags": ["baseline", "unit_test"], "uid": "1"}]}} diff --git a/testing/references/bestest_hydronic/submit.json b/testing/references/bestest_hydronic/submit.json index 382f2169f..b9914bbf1 100644 --- a/testing/references/bestest_hydronic/submit.json +++ b/testing/references/bestest_hydronic/submit.json @@ -1 +1 @@ -{"dash_url": "https://api.boptest.net:8081//api/results", "payload": {"results": [{"account": {"apiKey": "valid_api_key"}, "boptestVersion": "0.x.x\n", "buildingType": {"uid": "bestest_hydronic"}, "controlStep": "86400.0", "dateRun": "2020-05-17 00:00:00", "forecastParameters": {"horizon": 3600.0, "interval": 300.0}, "isShared": true, "kpis": {"cost_tot": 0.465938501176076, "emis_tot": 1.628640744152675, "ener_tot": 9.000706489138203, "idis_tot": 0.0, "pdih_tot": null, "pele_tot": 0.00025517153990852034, "pgas_tot": 0.11798039028403248, "tdis_tot": 18.217583154564775, "time_rat": 0}, "scenario": {"electricityPrice": "dynamic", "timePeriod": "peak_heat_day", "weatherForecastUncertainty": "deterministic"}, "tags": ["baseline", "unit_test"], "uid": "1"}]}} +{"dash_url": "https://api.boptest.net:8081//api/results", "payload": {"results": [{"account": {"apiKey": "valid_api_key"}, "boptestVersion": "0.3.0\n", "buildingType": {"uid": "bestest_hydronic"}, "controlStep": "86400.0", "dateRun": "2020-05-17 00:00:00", "forecastParameters": {"horizon": 3600.0, "interval": 300.0}, "isShared": true, "kpis": {"cost_tot": 0.465938501176076, "emis_tot": 1.628640744152675, "ener_tot": 9.000706489138203, "idis_tot": 0.0, "pdih_tot": null, "pele_tot": 0.00025517153990852034, "pgas_tot": 0.11798039028403248, "tdis_tot": 18.217583154564775, "time_rat": 0}, "scenario": {"electricityPrice": "dynamic", "timePeriod": "peak_heat_day", "weatherForecastUncertainty": "deterministic"}, "tags": ["baseline", "unit_test"], "uid": "1"}]}} diff --git a/testing/references/bestest_hydronic_heat_pump/submit.json b/testing/references/bestest_hydronic_heat_pump/submit.json index d5097d5f9..d3515fa9b 100644 --- a/testing/references/bestest_hydronic_heat_pump/submit.json +++ b/testing/references/bestest_hydronic_heat_pump/submit.json @@ -1 +1 @@ -{"dash_url": "https://api.boptest.net:8081//api/results", "payload": {"results": [{"account": {"apiKey": "valid_api_key"}, "boptestVersion": "0.x.x\n", "buildingType": {"uid": "bestest_hydronic_heat_pump"}, "controlStep": "86400.0", "dateRun": "2020-05-17 00:00:00", "forecastParameters": {"horizon": 3600.0, "interval": 300.0}, "isShared": true, "kpis": {"cost_tot": 0.8828705411678542, "emis_tot": 0.5808444758926209, "ener_tot": 3.4781106340875496, "idis_tot": 0.0, "pdih_tot": null, "pele_tot": 0.018913031427716383, "pgas_tot": null, "tdis_tot": 8.382467492017371, "time_rat": 0}, "scenario": {"electricityPrice": "dynamic", "timePeriod": "peak_heat_day", "weatherForecastUncertainty": "deterministic"}, "tags": ["baseline", "unit_test"], "uid": "1"}]}} +{"dash_url": "https://api.boptest.net:8081//api/results", "payload": {"results": [{"account": {"apiKey": "valid_api_key"}, "boptestVersion": "0.3.0\n", "buildingType": {"uid": "bestest_hydronic_heat_pump"}, "controlStep": "86400.0", "dateRun": "2020-05-17 00:00:00", "forecastParameters": {"horizon": 3600.0, "interval": 300.0}, "isShared": true, "kpis": {"cost_tot": 0.8828705411678542, "emis_tot": 0.5808444758926209, "ener_tot": 3.4781106340875496, "idis_tot": 0.0, "pdih_tot": null, "pele_tot": 0.018913031427716383, "pgas_tot": null, "tdis_tot": 8.382467492017371, "time_rat": 0}, "scenario": {"electricityPrice": "dynamic", "timePeriod": "peak_heat_day", "weatherForecastUncertainty": "deterministic"}, "tags": ["baseline", "unit_test"], "uid": "1"}]}} diff --git a/testing/references/multizone_office_simple_air/submit.json b/testing/references/multizone_office_simple_air/submit.json index 8f99419d9..010cfccf7 100644 --- a/testing/references/multizone_office_simple_air/submit.json +++ b/testing/references/multizone_office_simple_air/submit.json @@ -1 +1 @@ -{"dash_url": "https://api.boptest.net:8081//api/results", "payload": {"results": [{"account": {"apiKey": "valid_api_key"}, "boptestVersion": "0.x.x\n", "buildingType": {"uid": "multizone_office_simple_air"}, "controlStep": "86400.0", "dateRun": "2020-05-17 00:00:00", "forecastParameters": {"horizon": 3600.0, "interval": 300.0}, "isShared": true, "kpis": {"cost_tot": 0.16537569781601175, "emis_tot": 0.6104861553474377, "ener_tot": 1.790281980491019, "idis_tot": 0.0, "pdih_tot": null, "pele_tot": 0.03334238212125819, "pgas_tot": null, "tdis_tot": 11.835569616547645, "time_rat": 0}, "scenario": {"electricityPrice": "dynamic", "timePeriod": "peak_heat_day", "weatherForecastUncertainty": "deterministic"}, "tags": ["baseline", "unit_test"], "uid": "1"}]}} +{"dash_url": "https://api.boptest.net:8081//api/results", "payload": {"results": [{"account": {"apiKey": "valid_api_key"}, "boptestVersion": "0.3.0\n", "buildingType": {"uid": "multizone_office_simple_air"}, "controlStep": "86400.0", "dateRun": "2020-05-17 00:00:00", "forecastParameters": {"horizon": 3600.0, "interval": 300.0}, "isShared": true, "kpis": {"cost_tot": 0.16537569781601175, "emis_tot": 0.6104861553474377, "ener_tot": 1.790281980491019, "idis_tot": 0.0, "pdih_tot": null, "pele_tot": 0.03334238212125819, "pgas_tot": null, "tdis_tot": 11.835569616547645, "time_rat": 0}, "scenario": {"electricityPrice": "dynamic", "timePeriod": "peak_heat_day", "weatherForecastUncertainty": "deterministic"}, "tags": ["baseline", "unit_test"], "uid": "1"}]}} diff --git a/testing/references/multizone_residential_hydronic/submit.json b/testing/references/multizone_residential_hydronic/submit.json index 697df510d..b4acb8191 100644 --- a/testing/references/multizone_residential_hydronic/submit.json +++ b/testing/references/multizone_residential_hydronic/submit.json @@ -1 +1 @@ -{"dash_url": "https://api.boptest.net:8081//api/results", "payload": {"results": [{"account": {"apiKey": "valid_api_key"}, "boptestVersion": "0.x.x\n", "buildingType": {"uid": "multizone_residential_hydronic"}, "controlStep": "86400.0", "dateRun": "2020-05-17 00:00:00", "forecastParameters": {"horizon": 3600.0, "interval": 300.0}, "isShared": true, "kpis": {"cost_tot": 0.7912501281574299, "emis_tot": 1.4253043873549793, "ener_tot": 8.140085161898376, "idis_tot": 9114.593818178417, "pdih_tot": null, "pele_tot": 0.0017390231869758264, "pgas_tot": 0.09592720495532536, "tdis_tot": 22.003355978645242, "time_rat": 0}, "scenario": {"electricityPrice": "dynamic", "timePeriod": "peak_heat_day", "weatherForecastUncertainty": "deterministic"}, "tags": ["baseline", "unit_test"], "uid": "1"}]}} +{"dash_url": "https://api.boptest.net:8081//api/results", "payload": {"results": [{"account": {"apiKey": "valid_api_key"}, "boptestVersion": "0.3.0\n", "buildingType": {"uid": "multizone_residential_hydronic"}, "controlStep": "86400.0", "dateRun": "2020-05-17 00:00:00", "forecastParameters": {"horizon": 3600.0, "interval": 300.0}, "isShared": true, "kpis": {"cost_tot": 0.7912501281574299, "emis_tot": 1.4253043873549793, "ener_tot": 8.140085161898376, "idis_tot": 9114.593818178417, "pdih_tot": null, "pele_tot": 0.0017390231869758264, "pgas_tot": 0.09592720495532536, "tdis_tot": 22.003355978645242, "time_rat": 0}, "scenario": {"electricityPrice": "dynamic", "timePeriod": "peak_heat_day", "weatherForecastUncertainty": "deterministic"}, "tags": ["baseline", "unit_test"], "uid": "1"}]}} diff --git a/testing/references/singlezone_commercial_hydronic/submit.json b/testing/references/singlezone_commercial_hydronic/submit.json index 03411c22e..72a86d012 100644 --- a/testing/references/singlezone_commercial_hydronic/submit.json +++ b/testing/references/singlezone_commercial_hydronic/submit.json @@ -1 +1 @@ -{"dash_url": "https://api.boptest.net:8081//api/results", "payload": {"results": [{"account": {"apiKey": "valid_api_key"}, "boptestVersion": "0.x.x\n", "buildingType": {"uid": "singlezone_commercial_hydronic"}, "controlStep": "86400.0", "dateRun": "2020-05-17 00:00:00", "forecastParameters": {"horizon": 3600.0, "interval": 300.0}, "isShared": true, "kpis": {"cost_tot": 0.26318819272610183, "emis_tot": 0.3835824174014755, "ener_tot": 3.216803183654829, "idis_tot": 5.423240054209877, "pdih_tot": 0.08966901817018418, "pele_tot": 0.004907824412797784, "pgas_tot": null, "tdis_tot": 7.9949290180759505, "time_rat": 0}, "scenario": {"electricityPrice": "dynamic", "timePeriod": "peak_heat_day", "weatherForecastUncertainty": "deterministic"}, "tags": ["baseline", "unit_test"], "uid": "1"}]}} +{"dash_url": "https://api.boptest.net:8081//api/results", "payload": {"results": [{"account": {"apiKey": "valid_api_key"}, "boptestVersion": "0.3.0\n", "buildingType": {"uid": "singlezone_commercial_hydronic"}, "controlStep": "86400.0", "dateRun": "2020-05-17 00:00:00", "forecastParameters": {"horizon": 3600.0, "interval": 300.0}, "isShared": true, "kpis": {"cost_tot": 0.26318819272610183, "emis_tot": 0.3835824174014755, "ener_tot": 3.216803183654829, "idis_tot": 5.423240054209877, "pdih_tot": 0.08966901817018418, "pele_tot": 0.004907824412797784, "pgas_tot": null, "tdis_tot": 7.9949290180759505, "time_rat": 0}, "scenario": {"electricityPrice": "dynamic", "timePeriod": "peak_heat_day", "weatherForecastUncertainty": "deterministic"}, "tags": ["baseline", "unit_test"], "uid": "1"}]}} diff --git a/testing/references/testcase1/submit.json b/testing/references/testcase1/submit.json index f826d7abe..26139dd9f 100644 --- a/testing/references/testcase1/submit.json +++ b/testing/references/testcase1/submit.json @@ -1 +1 @@ -{"dash_url": "https://api.boptest.net:8081//api/results", "payload": {"results": [{"account": {"apiKey": "valid_api_key"}, "boptestVersion": "0.x.x\n", "buildingType": {"uid": "testcase1"}, "controlStep": "86400.0", "dateRun": "2020-05-17 00:00:00", "forecastParameters": {"horizon": 3600.0, "interval": 300.0}, "isShared": true, "kpis": {"cost_tot": 0.8993115895520294, "emis_tot": 2.5694616844343696, "ener_tot": 12.847308422171846, "idis_tot": 7118.611500670603, "pdih_tot": null, "pele_tot": null, "pgas_tot": 0.11536491775500209, "tdis_tot": 503.7616200965347, "time_rat": 0}, "scenario": {"electricityPrice": "dynamic", "timePeriod": "test_day", "weatherForecastUncertainty": "deterministic"}, "tags": ["baseline", "unit_test"], "uid": "1"}]}} +{"dash_url": "https://api.boptest.net:8081//api/results", "payload": {"results": [{"account": {"apiKey": "valid_api_key"}, "boptestVersion": "0.3.0\n", "buildingType": {"uid": "testcase1"}, "controlStep": "86400.0", "dateRun": "2020-05-17 00:00:00", "forecastParameters": {"horizon": 3600.0, "interval": 300.0}, "isShared": true, "kpis": {"cost_tot": 0.8993115895520294, "emis_tot": 2.5694616844343696, "ener_tot": 12.847308422171846, "idis_tot": 7118.611500670603, "pdih_tot": null, "pele_tot": null, "pgas_tot": 0.11536491775500209, "tdis_tot": 503.7616200965347, "time_rat": 0}, "scenario": {"electricityPrice": "dynamic", "timePeriod": "test_day", "weatherForecastUncertainty": "deterministic"}, "tags": ["baseline", "unit_test"], "uid": "1"}]}} diff --git a/testing/references/testcase2/submit.json b/testing/references/testcase2/submit.json index e2468be4e..a0f476416 100644 --- a/testing/references/testcase2/submit.json +++ b/testing/references/testcase2/submit.json @@ -1 +1 @@ -{"dash_url": "https://api.boptest.net:8081//api/results", "payload": {"results": [{"account": {"apiKey": "valid_api_key"}, "boptestVersion": "0.x.x\n", "buildingType": {"uid": "testcase2"}, "controlStep": "86400.0", "dateRun": "2020-05-17 00:00:00", "forecastParameters": {"horizon": 3600.0, "interval": 300.0}, "isShared": true, "kpis": {"cost_tot": 1.5461632218795267, "emis_tot": 5.779914422546377, "ener_tot": 11.559828845092754, "idis_tot": 0.0, "pdih_tot": null, "pele_tot": 0.08727961692454471, "pgas_tot": null, "tdis_tot": 0.587950393326939, "time_rat": 0}, "scenario": {"electricityPrice": "dynamic", "timePeriod": "test_day", "weatherForecastUncertainty": "deterministic"}, "tags": ["baseline", "unit_test"], "uid": "1"}]}} +{"dash_url": "https://api.boptest.net:8081//api/results", "payload": {"results": [{"account": {"apiKey": "valid_api_key"}, "boptestVersion": "0.3.0\n", "buildingType": {"uid": "testcase2"}, "controlStep": "86400.0", "dateRun": "2020-05-17 00:00:00", "forecastParameters": {"horizon": 3600.0, "interval": 300.0}, "isShared": true, "kpis": {"cost_tot": 1.5461632218795267, "emis_tot": 5.779914422546377, "ener_tot": 11.559828845092754, "idis_tot": 0.0, "pdih_tot": null, "pele_tot": 0.08727961692454471, "pgas_tot": null, "tdis_tot": 0.587950393326939, "time_rat": 0}, "scenario": {"electricityPrice": "dynamic", "timePeriod": "test_day", "weatherForecastUncertainty": "deterministic"}, "tags": ["baseline", "unit_test"], "uid": "1"}]}} diff --git a/testing/references/testcase3/submit.json b/testing/references/testcase3/submit.json index 69fad8e12..53d6224a6 100644 --- a/testing/references/testcase3/submit.json +++ b/testing/references/testcase3/submit.json @@ -1 +1 @@ -{"dash_url": "https://api.boptest.net:8081//api/results", "payload": {"results": [{"account": {"apiKey": "valid_api_key"}, "boptestVersion": "0.x.x\n", "buildingType": {"uid": "testcase3"}, "controlStep": "86400.0", "dateRun": "2020-05-17 00:00:00", "forecastParameters": {"horizon": 3600.0, "interval": 300.0}, "isShared": true, "kpis": {"cost_tot": 0.9675513120754404, "emis_tot": 2.764432320215544, "ener_tot": 13.822161601077719, "idis_tot": 3606.22674562425, "pdih_tot": null, "pele_tot": null, "pgas_tot": 0.12017492256506475, "tdis_tot": 443.50197076537665, "time_rat": 0}, "scenario": {"electricityPrice": "dynamic", "timePeriod": "test_day", "weatherForecastUncertainty": "deterministic"}, "tags": ["baseline", "unit_test"], "uid": "1"}]}} +{"dash_url": "https://api.boptest.net:8081//api/results", "payload": {"results": [{"account": {"apiKey": "valid_api_key"}, "boptestVersion": "0.3.0\n", "buildingType": {"uid": "testcase3"}, "controlStep": "86400.0", "dateRun": "2020-05-17 00:00:00", "forecastParameters": {"horizon": 3600.0, "interval": 300.0}, "isShared": true, "kpis": {"cost_tot": 0.9675513120754404, "emis_tot": 2.764432320215544, "ener_tot": 13.822161601077719, "idis_tot": 3606.22674562425, "pdih_tot": null, "pele_tot": null, "pgas_tot": 0.12017492256506475, "tdis_tot": 443.50197076537665, "time_rat": 0}, "scenario": {"electricityPrice": "dynamic", "timePeriod": "test_day", "weatherForecastUncertainty": "deterministic"}, "tags": ["baseline", "unit_test"], "uid": "1"}]}} diff --git a/version.txt b/version.txt index b6ad2c20f..0d91a54c7 100644 --- a/version.txt +++ b/version.txt @@ -1 +1 @@ -0.x.x +0.3.0