Feature: Add FLORIS tuning (#115)

* Initiation of branch * Lower the noise on this example * Incremental progress * Update noteboook * Add emgauss models * Add support changing fi parameters within call * sort imports * small updates * small changes * Sort imports * Small cleaning * Add a function to replicate nan values * Add a test of replicating nan values * Add docstring * Remove redundant class * Add test script * Implement more of tuning script * Update notebook * Update test script * Update utils * Update tuner * Add a tuning notebook * Add option for absolute uplift * Use absolute uplift * Chain through absolute uplift * Updates * updates * Update notebook * refactoring * Building out pseudocode for the wake loss eval case. * Move step of generating df_fi into the specific evaluators, if required. * setting up floris candidates. * Commit to remerge develop. * small refactoring * refactoring * A little progress * Small edits * updates * fix check call * re-run * Updates * some clean up * Starting smarteole version * yaw right side * yaw one side * One way steering applied. * Typo fix. * Updated emgauss defaults; artificial tuning notebook running as expected. * Adding small readme. * Point users to existing pp fitting. * Removing sklearn dependency. * Fix yaw angles * Delete floris tuner * Update examples * Rename artificial example * Update examples --------- Co-authored-by: misi9170 <[email protected]>
NREL · Oct 26, 2023 · b66ea70 · b66ea70
1 parent 9cfdbd0
commit b66ea70
Show file tree

Hide file tree

Showing 15 changed files with 3,568 additions and 99 deletions.
diff --git a/README.rst b/README.rst
@@ -1,18 +1,21 @@
-=============================================
-FLORIS-based Analysis for SCADA data (FLASC)
-=============================================
+==========
+FLORIS-Based Analysis for SCADA Data (FLASC)
+==========
 
-**Further documentation is available at http://flasc.readthedocs.io/.**
+**Note:** Further documentation is available at **http://flasc.readthedocs.io/.**
 
-FLASC provides a rich suite of analysis tools for SCADA data
-filtering & analysis, wind farm model validation, field experiment design,
-and field experiment monitoring. The repository is centrally built around
-NRELs in-house `floris` wind farm model, available at
-https://github.com/nrel/floris. FLASC also largely relies on the `energy ratio`
-to, among others, quantify wake losses in synthetic and historical data, to
-perform turbine northing calibrations, and for model parameter estimation.
+Description
+-----------
 
-For technical questions, please send an email to [email protected].
+FLASC provides a **rich suite of analysis tools** for SCADA data filtering, analysis, 
+wind farm model validation, field experiment design, and field experiment monitoring. 
+
+The repository is centrally built around NRELs in-house ``FLORIS`` wind farm model, available at
+**https://github.com/nrel/floris**. FLASC also largely relies on the ``energy ratio``, among others, 
+to quantify wake losses in synthetic and historical data, to perform turbine northing calibrations, 
+and model parameter estimation.
+
+For technical questions or concerns, please email [email protected].
 
 .. image:: https://readthedocs.org/projects/flasc/badge/?version=main
    :target: http://flasc.readthedocs.io/
@@ -36,22 +39,33 @@ For technical questions, please send an email to [email protected].
 
 Installation
 ------------
+
 We recommend installing this repository in a separate virtual environment.
 After creating a new virtual environment, clone this repository to your local
 system and install it locally using ``pip``. The command for this is ``pip install -e flasc``.
 
 Features
 --------
-FLASC consists of multiple modules, including:
 
-* *flasc.dataframe_operations*: This module includes functionality to easily manipulate pandas data frames. Functions include filtering data by wind direction, wind speed an/or TI, deriving the ambient conditions from the upstream turbines, all the while dealing with angle wrapping for angular variables.
-* *flasc.energy_ratio*: this module contains classes to calculate and visualize the energy ratio as defined by Fleming et al. (2019). The energy ratio is a very useful quantity in SCADA data analysis and related model	validation. It represents the amount of energy produced by a turbine relative to what that turbine would have produced if no wakes were present. Various classes are included in this model, from classes used to calculate and plot the energy ratio for a single dataset, a class for multiple datasets, and a class that calculates the wind direction bias for every turbine by maximizing the energy ratio fit between FLORIS and SCADA data. Various visualization methods are included such as energy ratio plots and automated generation of detailed excel spreadsheets to determine where and which turbines performed differently than expected. These methods	can be used both for model validation and for processing field campaign data, e.g. baseline vs optimized operation.
-* *flasc.floris_tools*: this module contains functions that leverage the floris model directly. This includes functions to calculate a large set of floris simulations (with MPI, optionally) for different atmospheric conditions, yaw misalignments and/or model parameters. It also includes two functions	to precalculate and respectively interpolate from a large set of model solutions to speed up further postprocessing.
-* *flasc.model_estimation*: This is a module related to the estimation of parameters in the floris wind farm model. One class herein, called `floris_sensitivity_analysis`, performs Sobol parameter sensitivity	studies to determine which parameters are most sensitive in various situations (atmospheric conditions, turbine settings, wind farm layouts).
-* *flasc.optimization*: The optimization module includes functions to estimate the timeshift between two sources of data, for example, to sychronize measurements from a met mast with measurements from SCADA data. The module also includes a function to estimate the offset between two timeseries of wind direction measurements. This is useful to determine the northing bias of a turbine if you know the correct calibration of at least one other wind turbine. Finally, this module also contains a function to estimate the atmospheric turbulence intensity based on the power measurements of the turbines inside a wind farm.
-* *flasc.raw_data_handling*: this module contains functions that supports importing and processing raw SCADA data files. Specifically, it provides a class called "sql_database_manager" which can be used to up- and download data between your local system and a remote SQL database. This class also  contains a GUI to visualize data existent in the remote repository. This repository also includes data handling for very large datasets. Data is saved in feather format for optimal balance of storage size and load/write speed. Additionally, can split one large dataframe into multiple dataframes and feather files.
-* *flasc.time_operations*: This module allows the user to easily downsample, upsample and calculate moving averages of a data frame with SCADA and/or FLORIS data. These functions allow the user to specify which columns contain angular variables, and consequently 360 deg wrapping is taken care	of. It also allows the user to calculate the min, max, std and median for downsampled data frames. It leverages efficient functions inherent in	pandas to maximize performance.
-* *flasc.turbine_analysis*: this module allows the user to analyze SCADA data on a turbine level. Outliers can be detected and removed. Filtering methods include sensor-stuck type of fault detection and analysis of the turbine wind speed-power curve.
+FLASC consists of several modules, including:
+
+* **flasc.dataframe_operations**: Contains functions to easily manipulate Pandas dataframes. This includes functions for filtering data by wind direction, wind speed and/or TI, and deriving the ambient conditions from the upstream turbines, all while handling angle wrapping for angular variables.
+
+* **flasc.energy_ratio**: Contains classes to calculate and visualize the *energy ratio* as defined by Fleming et al. (2019). The ``energy ratio`` is an important quantity in SCADA data analysis and related model validation. It represents the amount of energy produced by a turbine *relative* to what that turbine would have produced if *no* wakes were present. Various classes are included in this model, such as classes for calculating and plotting the energy ratio for single or multiple datasets, and a class that calculates the wind direction bias for every turbine by maximizing the energy ratio fit between FLORIS and SCADA data. There are also visualization functions for energy ratio plots and automated generation of detailed excel spreadsheets to determine where and which turbines performed differently than expected. These functions can be used both for model validation and processing field campaign data (e.g. baseline vs. controlled operation).
+
+* **flasc.floris_tools**: Contains functions that leverage the FLORIS model *directly*. This includes functions to calculate a large set of FLORIS simulations (with MPI, optionally) for different atmospheric conditions, yaw misalignments, and/or model parameters. There are also functions to pre-calculate and respectively interpolate from a large dataset of model solutions to improve the efficiency of further post-processing.
+
+* **flasc.model_estimation**: Contains classes pertaining to the estimation of parameters in the FLORIS wind farm model. The class ``floris_sensitivity_analysis``, performs Sobol parameter sensitivity studies to determine which parameters are most sensitive in certain situations (e.g. atmospheric conditions, turbine settings, wind farm layouts).
+
+* **flasc.optimization**: Contains functions that estimate the time shift between two sources of data, for example, to synchronize measurements from a met mast with measurements from SCADA data. Includes a function to estimate the offset between two timeseries of wind direction measurements. This is useful to determine the northing bias of a turbine assuming the user has knowledge about the correct calibration of at *least* one turbine. Also includes a function to estimate the atmospheric turbulence intensity based on the power measurements of the turbines inside a wind farm.
+
+* **flasc.raw_data_handling**: Contains functions that support importing and processing raw SCADA data files. The class ``sql_database_manager`` up/downloads data between the local file system and a remote SQL database. This class also contains a GUI to visualize data existent in the remote repository. This repository also includes data handling for very large datasets. Data is saved using feather format for an *optimal* balance of storage size and read/write speed. Additionally, large dataframes can be *split* up into multiple dataframes and feather files.
+
+* **flasc.time_operations**: Contains functions for downsampling, upsampling and calculating moving averages of a dataframe containing SCADA and/or FLORIS data. These functions allow the user to specify which columns contain angular variables, and consequently handle 360 deg wrapping. Additionally, these functions can calculate the min, max, std and median for downsampled dataframes. This module leverages efficient functions *native* to Pandas to maximize performance.
+
+* **flasc.turbine_analysis**: Contains functions that enable SCADA data analysis on the *turbine* level. Outliers can be detected and removed. Filtering-specific functions include sensor-stuck type fault detection and analysis of the turbine wind speed-power curve.
+
+* **flasc.model_tuning**: Contains a class for tuning FLORIS model parameters to yield results that *align* with SCADA data. This class offers a suite of functions for tuning, visualizations that compare FLORIS to SCADA and outputting the tuned parameters to a YAML file for later use. 
 
 License
 ------------
@@ -60,7 +74,7 @@ Licensed under the Apache License, Version 2.0 (the "License");
 you may not use this file except in compliance with the License.
 You may obtain a copy of the License at
 
-   http://www.apache.org/licenses/LICENSE-2.0
+   **http://www.apache.org/licenses/LICENSE-2.0**
 
 Unless required by applicable law or agreed to in writing, software
 distributed under the License is distributed on an "AS IS" BASIS,

diff --git a/examples_artificial_data/04_floris_tuning/00_tune_floris_to_artificial_data.ipynb b/examples_artificial_data/04_floris_tuning/00_tune_floris_to_artificial_data.ipynb
diff --git a/examples_artificial_data/floris_input_artificial/emgauss.yaml b/examples_artificial_data/floris_input_artificial/emgauss.yaml
@@ -0,0 +1,203 @@
+
+###
+# A name for this input file.
+# This is not currently only for the user's reference.
+name: GCH
+
+###
+# A description of the contents of this input file.
+# This is not currently only for the user's reference.
+description: Three turbines using Gauss Curl Hybrid model
+
+###
+# The earliest verion of FLORIS this input file supports.
+# This is not currently only for the user's reference.
+floris_version: v3.0.0
+
+###
+# Configure the logging level and where to show the logs.
+logging:
+
+  ###
+  # Settings for logging to the console (i.e. terminal).
+  console:
+
+    ###
+    # Can be "true" or "false".
+    enable: true
+
+    ###
+    # Set the severity to show output. Messages at this level or higher will be shown.
+    # Can be one of "CRITICAL", "ERROR", "WARNING", "INFO", "DEBUG".
+    level: WARNING
+
+  ###
+  # Settings for logging to a file.
+  file:
+
+    ###
+    # Can be "true" or "false".
+    enable: false
+
+    ###
+    # Set the severity to show output. Messages at this level or higher will be shown.
+    # Can be one of "CRITICAL", "ERROR", "WARNING", "INFO", "DEBUG".
+    level: WARNING
+
+###
+# Configure the solver for the type of simulation.
+solver:
+
+  ###
+  # Select the solver type.
+  # Can be one of: "turbine_grid", "flow_field_grid", "flow_field_planar_grid".
+  type: turbine_grid
+
+  ###
+  # Options for the turbine type selected above. See the solver documentation for available parameters.
+  turbine_grid_points: 3
+
+###
+# Configure the turbine types and their placement within the wind farm.
+farm:
+
+  ###
+  # Coordinates for the turbine locations in the x-direction which is typically considered
+  # to be the streamwise direction (left, right) when the wind is out of the west.
+  # The order of the coordinates here corresponds to the index of the turbine in the primary
+  # data structures.
+  layout_x:
+  - 0.0
+  - 630.0
+  - 1260.0
+
+  ###
+  # Coordinates for the turbine locations in the y-direction which is typically considered
+  # to be the spanwise direction (up, down) when the wind is out of the west.
+  # The order of the coordinates here corresponds to the index of the turbine in the primary
+  # data structures.
+  layout_y:
+  - 0.0
+  - 0.0
+  - 0.0
+
+  ###
+  # Listing of turbine types for placement at the x and y coordinates given above.
+  # The list length must be 1 or the same as ``layout_x`` and ``layout_y``. If it is a
+  # single value, all turbines are of the same type. Otherwise, the turbine type
+  # is mapped to the location at the same index in ``layout_x`` and ``layout_y``.
+  # The types can be either a name included in the turbine_library or
+  # a full definition of a wind turbine directly.
+  turbine_type:
+  - nrel_5MW
+
+###
+# Configure the atmospheric conditions.
+flow_field:
+
+  ###
+  # Air density.
+  air_density: 1.225
+
+  ###
+  # The height to consider the "center" of the vertical wind speed profile
+  # due to shear. With a shear exponent not 1, the wind speed at this height
+  # will be the value given in ``wind_speeds``. Above and below this height,
+  # the wind speed will change according to the shear profile; see
+  # :py:meth:`.FlowField.initialize_velocity_field`.
+  # For farms consisting of one wind turbine type, use ``reference_wind_height: -1``
+  # to use the hub height of the wind turbine definition. For multiple wind turbine
+  # types, the reference wind height must be given explicitly.
+  reference_wind_height: -1
+
+  ###
+  # The level of turbulence intensity level in the wind.
+  turbulence_intensity: 0.06
+
+  ###
+  # The wind directions to include in the simulation.
+  # 0 is north and 270 is west.
+  wind_directions:
+  - 270.0
+
+  ###
+  # The exponent used to model the wind shear profile; see
+  # :py:meth:`.FlowField.initialize_velocity_field`.
+  wind_shear: 0.12
+
+  ###
+  # The wind speeds to include in the simulation.
+  wind_speeds:
+  - 8.0
+
+  ###
+  # The wind veer as a constant value for all points in the grid.
+  wind_veer: 0.0
+
+###
+# Configure the wake model.
+wake:
+  model_strings:
+    combination_model: sosfs
+    deflection_model: empirical_gauss
+    turbulence_model: wake_induced_mixing
+    velocity_model: empirical_gauss
+
+  enable_secondary_steering: false
+  enable_yaw_added_recovery: true
+  enable_transverse_velocities: false
+
+  wake_deflection_parameters:
+    gauss:
+      ad: 0.0
+      alpha: 0.58
+      bd: 0.0
+      beta: 0.077
+      dm: 1.0
+      ka: 0.38
+      kb: 0.004
+    jimenez:
+      ad: 0.0
+      bd: 0.0
+      kd: 0.05
+    empirical_gauss:
+      horizontal_deflection_gain_D: 3.0
+      vertical_deflection_gain_D: -1
+      deflection_rate: 30
+      mixing_gain_deflection: 0.0
+      yaw_added_mixing_gain: 0.0
+
+  wake_velocity_parameters:
+    cc:
+      a_s: 0.179367259
+      b_s: 0.0118889215
+      c_s1: 0.0563691592
+      c_s2: 0.13290157
+      a_f: 3.11
+      b_f: -0.68
+      c_f: 2.41
+      alpha_mod: 1.0
+    gauss:
+      alpha: 0.58
+      beta: 0.077
+      ka: 0.38
+      kb: 0.004
+    jensen:
+      we: 0.05
+    empirical_gauss:
+      wake_expansion_rates:
+      - 0.023
+      - 0.008
+      breakpoints_D:
+      - 10
+      sigma_0_D: 0.28
+      smoothing_length_D: 2.0
+      mixing_gain_velocity: 2.0
+  wake_turbulence_parameters:
+    crespo_hernandez:
+      initial: 0.1
+      constant: 0.5
+      ai: 0.8
+      downstream: -0.32
+    wake_induced_mixing:
+      atmospheric_ti_gain: 0.0