-
Notifications
You must be signed in to change notification settings - Fork 415
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add a boundary layer module to estimate boundary height #3572
base: main
Are you sure you want to change the base?
Changes from 1 commit
a99c908
6bdf81b
106ac48
0f16c2d
c60a30b
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change | ||||
---|---|---|---|---|---|---|
@@ -0,0 +1,305 @@ | ||||||
#!/usr/bin/python | ||||||
# -*-coding:utf-8 -*- | ||||||
Check failure on line 2 in src/metpy/calc/boundarylayer.py GitHub Actions / Flake8
|
||||||
""" | ||||||
Check failure on line 3 in src/metpy/calc/boundarylayer.py GitHub Actions / Flake8
|
||||||
Contains a collection of boundary layer height estimations. | ||||||
|
||||||
|
||||||
References: | ||||||
Check failure on line 7 in src/metpy/calc/boundarylayer.py GitHub Actions / Flake8
Check failure on line 7 in src/metpy/calc/boundarylayer.py GitHub Actions / Flake8
|
||||||
----------- | ||||||
Check failure on line 8 in src/metpy/calc/boundarylayer.py GitHub Actions / Flake8
|
||||||
|
||||||
[Col14]: Collaud Coen, M., Praz, C., Haefele, A., Ruffieux, D., Kaufmann, P., and Calpini, B. (2014): | ||||||
Check failure on line 10 in src/metpy/calc/boundarylayer.py GitHub Actions / Flake8
|
||||||
Determination and climatology of the planetary boundary layer height above the Swiss plateau by in situ and remote sensing measurements as well as by the COSMO-2 model | ||||||
Atmos. Chem. Phys., 14, 13205–13221. | ||||||
|
||||||
[HL06]: Hennemuth, B., & Lammert, A. (2006): | ||||||
Determination of the atmospheric boundary layer height from radiosonde and lidar backscatter. | ||||||
Boundary-Layer Meteorology, 120(1), 181-200. | ||||||
|
||||||
[Guo16]: Guo, J., Miao, Y., Zhang, Y., Liu, H., Li, Z., Zhang, W., ... & Zhai, P. (2016): | ||||||
The climatology of planetary boundary layer height in China derived from radiosonde and reanalysis data. | ||||||
Atmos. Chem. Phys, 16(20), 13309-13319. | ||||||
|
||||||
[Sei00]: Seidel, D. J., Ao, C. O., & Li, K. (2010): | ||||||
Estimating climatological planetary boundary layer heights from radiosonde observations: Comparison of methods and uncertainty analysis. | ||||||
Journal of Geophysical Research: Atmospheres, 115(D16). | ||||||
|
||||||
[VH96]: Vogelezang, D. H. P., & Holtslag, A. A. M. (1996): | ||||||
Evaluation and model impacts of alternative boundary-layer height formulations. | ||||||
Boundary-Layer Meteorology, 81(3-4), 245-269. | ||||||
""" | ||||||
|
||||||
import numpy as np | ||||||
from copy import deepcopy | ||||||
|
||||||
import metpy.calc as mpcalc | ||||||
import metpy.constants as mpconsts | ||||||
from metpy.units import units | ||||||
|
||||||
|
||||||
def smooth(val, span): | ||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. XArray calls this a rolling mean. So does pandas. These would likely work better in the There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Thanks for the references! I knew some equivalent functions were already existing but they are not quite exactly the same (Xarray works on xarray.Dataset, Scipy has a slightly different strategy at the edges) and, given that the function is simple enough, it was less work to write it than to look for the existing one. Bottleneck's function seems to do exactly what I want but it is not listed in the Metpy's dependencies. Do you think it's worth adding it so I can use their moving-mean function? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. You'd have to ask one of the maintainers, but, in the meantime, would this be faster? cumulative_sums = np.nancumsum(val)
rolling_sums = cumulative_sums[span:] - cumulative_sums[:-span]
valid_index = np.isfinite(val)
cumulative_count = np.cumsum(valid_index)
rolling_count = cumulative_count[span:] - cumulative_count[:-span]
rolling_means = rolling_sums / rolling_count You'd need to pre-allocate Alternately, use SciPy for the bulk of the calculation, then re-do the edges the way you want. It would probably be a good idea to check whether this takes enough time that it's worth optimizing before going too far, though (as you may have noticed, I am not good at that). There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. With the testing data I have it's almost instantaneous and, as I moved other topics, I don't really have something bigger to quickly try it on. I suggest we leave it that way for now and other users might open another issue if when need to speed it up. Would that be alright? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. You picked the same name as elsewhere in MetPy, though the edge handling is again different from what you do here (they do not smooth close to the edge) and from SciPy, and they do not use The bigger test data would likely be someone trying to find boundary layer height from model data somewhere, and it looks like your code is designed for a profile at a time, not arrays of profiles (either the N x Z that description implies or the Z x Y x X conventional in model output). It might be a simple matter to add an |
||||||
"""Function that calculates the moving average with a given span. | ||||||
The span is given in number of points on which the average is made. | ||||||
|
||||||
Parameters | ||||||
---------- | ||||||
val: array-like | ||||||
Array of values | ||||||
span: int | ||||||
Span of the moving average. The higher the smoother | ||||||
|
||||||
Returns | ||||||
------- | ||||||
smoothed_val: array-like | ||||||
Array of smoothed values | ||||||
""" | ||||||
N = len(val) | ||||||
smoothed_val = deepcopy(val) | ||||||
for i in range(N): | ||||||
smoothed_val[i] = np.nanmean(val[i - min(span, i) : i + min(span, N - i)]) | ||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Suggested change
Would this be more clear as to the intent? |
||||||
|
||||||
return smoothed_val | ||||||
|
||||||
|
||||||
def bulk_richardson_number( | ||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Might close #628 |
||||||
height, | ||||||
potential_temperature, | ||||||
u, | ||||||
v, | ||||||
idxfoot: int = 0, | ||||||
ustar=0 * units.meter_per_second, | ||||||
): | ||||||
r"""Calculate the bulk Richardson number. | ||||||
|
||||||
See [VH96], eq. (3): | ||||||
|
||||||
.. math:: Ri = (g/\theta) * \frac{(\Delta z)(\Delta \theta)} | ||||||
{\left(\Delta u)^2 + (\Delta v)^2 + b(u_*)^2} | ||||||
|
||||||
Parameters | ||||||
---------- | ||||||
height : `pint.Quantity` | ||||||
Altitude (metres above ground) of the points in the profile | ||||||
potential_temperature : `pint.Quantity` | ||||||
Potential temperature profile | ||||||
u : `pint.Quantity` | ||||||
Zonal wind profile | ||||||
v : `pint.Quantity` | ||||||
Meridional wind profile | ||||||
idxfoot : int, optional | ||||||
The index of the foot point (first trusted measure), defaults to 0. | ||||||
|
||||||
Returns | ||||||
------- | ||||||
`pint.Quantity` | ||||||
Bulk Richardson number profile | ||||||
""" | ||||||
|
||||||
u[0] = 0 * units.meter_per_second | ||||||
v[0] = 0 * units.meter_per_second | ||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Is there a particular reason you force the winds at the first layer to zero without documenting the fact, rather than asserting an additional layer at the ground (below the profiles passed) where the winds are zero to satisfy the boundary conditions? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Not at all, I just didn't have the idea of adding an extra point. Thanks for the suggestion! The two lines would be replaced by:
Additionally, as the insertion of the ground changes the length of the profile, the returned profile should exclude it to keep the same size as the input ones:
Does it look OK with these changes? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Would you need to copy the point into the array? Could you just add a couple of variables for these? That would avoid the complication with the changing profile size, which would also impact if idxfoot == 0:
u0 = 0
v0 = 0
else:
u0 = u[idxfoot]
v0 = v[idxfoot]
theta0 = theta[idxfoot]
height0 = height[idxfoot] or even: if idxfoot == 0:
Du = u
Dv = v
else:
Du = u - u[idxfoot]
Dv = v - v[idxfoot] I'm not sure if height should be treated the same as velocity. I think I remember a displacement height that might complicate things? I think the main difference between this method and yours would be if the boundary layer was below the lowest passed level. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Your second suggestion:
looks the best to me. I will go for that one. Thanks! Regarding the problem that could arise with BL below the first level, I think the bulk Richardson method is out of scope anyway in this kind of situation, so I don't think we need to address it (rather refer to a more appropriate method). |
||||||
|
||||||
Dtheta = potential_temperature - potential_temperature[idxfoot] | ||||||
Du = u - u[idxfoot] | ||||||
Dv = v - v[idxfoot] | ||||||
Dz = height - height[idxfoot] | ||||||
|
||||||
idx0 = Du**2 + Dv**2 + ustar**2 == 0 | ||||||
if idx0.sum() > 0: | ||||||
bRi = np.ones_like(Dtheta) * np.nan * units.dimensionless | ||||||
bRi[~idx0] = ( | ||||||
(mpconsts.g / potential_temperature[~idx0]) | ||||||
* (Dtheta[~idx0] * Dz[~idx0]) | ||||||
/ (Du[~idx0] ** 2 + Dv[~idx0] ** 2 + ustar**2) | ||||||
) | ||||||
else: | ||||||
bRi = ( | ||||||
(mpconsts.g / potential_temperature) | ||||||
* (Dtheta * Dz) | ||||||
/ (Du**2 + Dv**2 + ustar**2) | ||||||
) | ||||||
|
||||||
return bRi | ||||||
|
||||||
|
||||||
def blh_from_richardson_bulk( | ||||||
height, | ||||||
potential_temperature, | ||||||
u, | ||||||
v, | ||||||
smoothingspan: int = 10, | ||||||
idxfoot: int = 0, | ||||||
bri_threshold=0.25 * units.dimensionless, | ||||||
ustar=0.1 * units.meter_per_second, | ||||||
): | ||||||
"""Calculate atmospheric boundary layer height with the method of | ||||||
bulk Richardson number. | ||||||
|
||||||
It is the height where the bulk Richardson number exceeds a given threshold. | ||||||
See [VH96, Sei00, Col14, Guo16]. | ||||||
|
||||||
Parameters | ||||||
---------- | ||||||
height : `pint.Quantity` | ||||||
Altitude (metres above ground) of the points in the profile | ||||||
potential_temperature : `pint.Quantity` | ||||||
Potential temperature profile | ||||||
u : `pint.Quantity` | ||||||
Zonal wind profile | ||||||
v : `pint.Quantity` | ||||||
Meridional wind profile | ||||||
smoothingspan : int, optional | ||||||
The amount of smoothing (number of points in moving average) | ||||||
idxfoot : int, optional | ||||||
The index of the foot point (first trusted measure), defaults to 0. | ||||||
bri_threshold : `pint.Quantity`, optional | ||||||
Threshold to exceed to get boundary layer top. Defaults to 0.25 | ||||||
ustar : `pint.Quantity`, optional | ||||||
Additional friction term in [VH96]. Defaluts to 0. | ||||||
|
||||||
Returns | ||||||
------- | ||||||
blh : `pint.Quantity` | ||||||
Boundary layer height estimation | ||||||
""" | ||||||
bRi = bulk_richardson_number( | ||||||
height, | ||||||
smooth(potential_temperature, smoothingspan), | ||||||
smooth(u, smoothingspan), | ||||||
smooth(v, smoothingspan), | ||||||
idxfoot=idxfoot, | ||||||
ustar=ustar, | ||||||
) | ||||||
|
||||||
height = height[~np.isnan(bRi)] | ||||||
bRi = bRi[~np.isnan(bRi)] | ||||||
|
||||||
if any(bRi > bri_threshold): | ||||||
iblh = np.where(bRi > bri_threshold)[0][0] | ||||||
blh = height[iblh] | ||||||
else: | ||||||
blh = np.nan * units.meter | ||||||
|
||||||
return blh | ||||||
|
||||||
|
||||||
def blh_from_parcel( | ||||||
height, | ||||||
potential_temperature, | ||||||
smoothingspan: int = 5, | ||||||
theta0=None, | ||||||
): | ||||||
"""Calculate atmospheric boundary layer height with the parcel method. | ||||||
|
||||||
It is the height where the potential temperature profile reaches its | ||||||
foot value. | ||||||
See [Sei00, HL06, Col14]. | ||||||
|
||||||
Parameters | ||||||
---------- | ||||||
height : `pint.Quantity` | ||||||
Altitude (metres above ground) of the points in the profile | ||||||
potential_temperature : `pint.Quantity` | ||||||
Potential temperature profile | ||||||
smoothingspan : int, optional | ||||||
The amount of smoothing (number of points in moving average) | ||||||
theta0 : `pint.Quantity`, optional | ||||||
Value of theta at the foot point (skip unstruted points or add extra term). If not provided, theta[0] is taken. | ||||||
|
||||||
Returns | ||||||
------- | ||||||
blh : `pint.Quantity` | ||||||
Boundary layer height estimation | ||||||
""" | ||||||
potential_temperature = smooth(potential_temperature, smoothingspan) | ||||||
|
||||||
if theta0 is None: | ||||||
theta0 = potential_temperature[0] | ||||||
|
||||||
if any(potential_temperature > theta0): | ||||||
iblh = np.where(potential_temperature > theta0)[0][0] | ||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. So this looks like a potential temperature threshold method. I would prefer "exceeds" over "reaches" in the documentation, given the usual description of the convective boundary layer. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Fine by me. I will also add that it's only suited for unstable boundary layer, as suggested in the main comment. The name of the method might vary with the authors, "parcel method" is the one I have seen the most, but I can include other names (e.g. "potential temperature threshold method") in the doc. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. There might be a difference in expectations if the boundary layer is saturated (i.e. fog or fair-weather cumulus), but describing alternate names should avoid that. |
||||||
blh = height[iblh] | ||||||
else: | ||||||
blh = np.nan * units.meter | ||||||
|
||||||
return blh | ||||||
|
||||||
|
||||||
def blh_from_humidity_gradient( | ||||||
height, | ||||||
humidity, | ||||||
smoothingspan: int = 5, | ||||||
idxfoot: int = 0, | ||||||
): | ||||||
"""Calculate atmospheric boundary layer height from the relative | ||||||
humidity gradient | ||||||
|
||||||
It is the height where the relative humidity or specific humidity gradient reaches a minimum. | ||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. As a side note, it should be possible to (ab)use this function with TKE or aerosol backscatter. I take it you considered wavelet methods out-of-scope? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Definitely possible to use it with TKE or aerosol backscatter. I don't mention it because the references I give are related to radiosounding data, while aerosol would come from lidar or ceilometer and TKE usually from model output, but they are very good way of deriving the BLH (especially the TKE, when turbulence is right in the model). The wavelet method is out of scope for this first PR, yes. It would be great to have it in the future but I will not have the time to commit for that. In the meantime, this gradient method + smoothing should be equivalent to the Haar wavelet. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
I think I've seen TKE from Doppler lidar. It's probably possible to derive something similar from Doppler radar, but the radar-based boundary-layer methods I've seen have been based on scattering/reflectivity.
Yeah, there's a lot of interesting things to do with boundary layers, and not all of them should be in a first attempt.
Fairly close, yes. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Sure, Doppler lidars can provide TKE but, as far as I remember, only the most powerful (and expensive) Doppler lidars will be able to have any signal from above the boundary layer, which make it difficult to run such an estimation because the top of the BL might be too close to the end of the profile. Anyway, the function is there, no harm trying it on TKE profiles! Do you think I should change the name (and the doc) of the function to make it more general? Something like There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
I think I've seen estimates based on SNR/signal quality in that situation, but likely less frequent of a use-case.
I like |
||||||
See [Sei00, HL06, Col14]. | ||||||
|
||||||
Parameters | ||||||
---------- | ||||||
height : `pint.Quantity` | ||||||
Altitude (metres above ground) of the points in the profile | ||||||
humidity : `pint.Quantity` | ||||||
Humidity (relative or specific) profile | ||||||
smoothingspan : int, optional | ||||||
The amount of smoothing (number of points in moving average) | ||||||
idxfoot : int, optional | ||||||
The index of the foot point (first trusted measure), defaults to 0. | ||||||
|
||||||
Returns | ||||||
------- | ||||||
blh : `pint.Quantity` | ||||||
Boundary layer height estimation | ||||||
""" | ||||||
|
||||||
dRHdz = mpcalc.first_derivative(smooth(humidity, smoothingspan), x=height) | ||||||
|
||||||
dRHdz = dRHdz[idxfoot:] | ||||||
height = height[idxfoot:] | ||||||
|
||||||
iblh = np.argmin(dRHdz) | ||||||
|
||||||
return height[iblh] | ||||||
|
||||||
|
||||||
def blh_from_temperature_inversion( | ||||||
height, | ||||||
temperature, | ||||||
smoothingspan: int = 5, | ||||||
idxfoot: int = 0, | ||||||
): | ||||||
"""Calculate atmospheric boundary layer height from the inversion of | ||||||
absolute temperature gradient | ||||||
|
||||||
It is the height where the temperature gradient (absolute or potential) changes of sign. | ||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Out of curiosity, how well does this work for the convective boundary layer with potential temperature? Or, for that matter, with the nocturnal stable boundary layer with either? I was expecting to see a threshold method on There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. From what I remember (my experience with this is now a bit old), this method is more suited for nocturnal stable boundary layers as it will track the end of the stable layer at the surface. For convective boundary layer, the parcel method should be preferred to this method, as this method would gives underestimated height with a large variability. The threshold of |
||||||
See [Col14]. | ||||||
|
||||||
Parameters | ||||||
---------- | ||||||
height : `pint.Quantity` | ||||||
Altitude (metres above ground) of the points in the profile | ||||||
humidity : `pint.Quantity` | ||||||
Temperature (absolute or potential) profile | ||||||
smoothingspan : int, optional | ||||||
The amount of smoothing (number of points in moving average) | ||||||
idxfoot : int, optional | ||||||
The index of the foot point (first trusted measure), defaults to 0. | ||||||
|
||||||
Returns | ||||||
------- | ||||||
blh : `pint.Quantity` | ||||||
Boundary layer height estimation | ||||||
""" | ||||||
|
||||||
dTdz = mpcalc.first_derivative(smooth(temperature, smoothingspan), x=height) | ||||||
|
||||||
dTdz = dTdz[idxfoot:] | ||||||
height = height[idxfoot:] | ||||||
|
||||||
if any(dTdz * dTdz[0] < 0): | ||||||
iblh = np.where(dTdz * dTdz[0] < 0)[0][0] | ||||||
blh = height[iblh] | ||||||
else: | ||||||
blh = np.nan * units.meter | ||||||
|
||||||
return blh |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I should probably ask the maintainers about one overall
References
section here with citations everywhere else vs. repeatedReferences
sections in each function that needs them.