Skip to content

Commit

Permalink
Merge branch 'main' into Ticket_23_Configure_Bluesky_Logging
Browse files Browse the repository at this point in the history
  • Loading branch information
Chsudeepta authored Oct 16, 2024
2 parents 59d1011 + a8bb972 commit 6cae0a2
Show file tree
Hide file tree
Showing 19 changed files with 1,815 additions and 1 deletion.
3 changes: 3 additions & 0 deletions doc/conf.py
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,8 @@
# -- General configuration ---------------------------------------------------
# https://www.sphinx-doc.org/en/master/usage/configuration.html#general-configuration

myst_enable_extensions = ["dollarmath"]

extensions = [
"myst_parser",
"sphinx.ext.autodoc",
Expand All @@ -46,3 +48,4 @@
html_static_path = ["_static"]

autoclass_content = "both"
myst_heading_anchors = 3
1 change: 1 addition & 0 deletions doc/dev/set_up_dev_environment.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,7 @@ python -m venv .venv
```
python -m pip install -e .[dev]
```
To run tests based on changes not yet in a release in the GUI PyDev console you will need to run this command using your global python install rather than in the venv.

## Unit tests
```
Expand Down
204 changes: 204 additions & 0 deletions doc/fitting/fitting.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,204 @@
# Fitting Callback

Similar to [`LivePlot`](../callbacks/plotting.md), `ibex_bluesky_core` provides a thin wrapper around Bluesky's LiveFit class, enhancing it with additional functionality to better support real-time data fitting. This wrapper not only offers a wide selection of models to fit your data on, but also introduces guess generation for fit parameters. As new data points are acquired, the wrapper refines these guesses dynamically, improving the accuracy of the fit with each additional piece of data, allowing for more efficient and adaptive real-time fitting workflows.

In order to use the wrapper, import `LiveFit` from `ibex_bluesky_core` rather than
`bluesky` directly:
```py
from ibex_bluesky_core.callbacks.fitting import LiveFit
```
**Note:** that you do not *need* `LivePlot` for `LiveFit` to work but it may be useful to know visaully how well the model fits to the raw data.

## Configuration

Below is a full example showing how to use standard `matplotlib` & `bluesky` functionality to apply fitting to a scan, using LivePlot and LiveFit. The fitting callback is set to expect data to take the form of a gaussian.
```py
import matplotlib.pyplot as plt
from ibex_bluesky_core.callbacks.plotting import LivePlot
from ibex_bluesky_core.callbacks.fitting import LiveFit
from bluesky.callbacks import LiveFitPlot

# Create a new figure to plot onto.
plt.figure()
# Make a new set of axes on that figure
ax = plt.gca()
# ax is shared by fit_callback and plot_callback

plot_callback = LivePlot(y="y_variable", x="x_variable", ax=ax)
fit_callback = LiveFit(Gaussian.fit(), y="y_variable", x="x_variable", update_every=0.5)
# update_every = in seconds, how often to recompute the fit. If `None`, do not compute until the end. Default is 1.
fit_plot_callback = LiveFitPlot(fit_callback, ax=ax, color="r")
```

**Note:** that the `LiveFit` callback doesn't directly do the plotting, it will return function parameters of the model its trying to fit to; a `LiveFit` object must be passed to `LiveFitPlot` which can then be subscribed to the `RunEngine`. See the [Bluesky Documentation](https://blueskyproject.io/bluesky/main/callbacks.html#livefitplot) for information on the various arguments that can be passed to the `LiveFitPlot` class.

The `plot_callback` and `fit_plot_callback` objects can then be subscribed to the `RunEngine`, using the same methods as described in [`LivePlot`](../callbacks/plotting.md). See the following example using `@subs_decorator`:

```py
@subs_decorator(
[
fit_plot_callback,
plot_callback
]
)

def plan() -> ...
```

## Models

We support **standard fits** for the following trends in data. See [Standard Fits](./standard_fits.md) for more infomation on the behaviour of these fits.

| Trend | Class Name in fitting_utils | Arguments |
| ----- | -------------------------| ----------|
| Linear | [Linear](./standard_fits.md#linear) | None |
| Polynomial | [Polynomial](./standard_fits.md#polynomial) | Polynomial Degree (int) |
| Gaussian | [Gaussian](./standard_fits.md#gaussian) | None |
| Lorentzian | [Lorentzian](./standard_fits.md#lorentzian) | None |
| Damped Oscillator | [DampedOsc](./standard_fits.md#damped-oscillator-dampedosc) | None |
| Slit Scan Fit | [SlitScan](./standard_fits.md#slit-scan-slitscan) | Max Slit Size (int) |
| Error Function | [ERF](./standard_fits.md#error-function-erf) | None |
| Complementary Error Function | [ERFC](./standard_fits.md/#complementary-error-function-erfc) | None |
| Top Hat | [TopHat](./standard_fits.md#top-hat-tophat) | None |
| Trapezoid | [Trapezoid](./standard_fits.md#trapezoid) | None |
| PeakStats (COM) **\*** | - | -

\* Native to Bluesky there is support for `PeakStats` which "computes peak statsitics after a run finishes." See [Bluesky docs](https://blueskyproject.io/bluesky/main/callbacks.html#peakstats) for more information on this. Similar to `LiveFit` and `LiveFitPLot`, `PeakStats` is a callback and must be passed to `PeakStatsPlot` to be plotted on a set of axes, which is subscribed to by the `RunEngine`.

-------

Each of the above fit classes has a `.fit()` which returns an object of type `FitMethod`. This tells `LiveFit` how to perform fitting on the data. `FitMethod` is defined in `ibex_bluesky_core.callbacks.fitting`.

There are *two* ways that you can choose how to fit a model to your data:

### Option 1: Use the standard fits
When only using the standard fits provided by `ibex_bluesky_core`, the following syntax can be used, replacing `[FIT]` with your chosen one from `ibex_bluesky_core.callbacks.fitting.fitting_utils`:

```py
from bluesky.callbacks import LiveFitPlot
from ibex_bluesky_core.callbacks.fitting.fitting_utils import [FIT]

# Pass [FIT].fit() to the first parameter of LiveFit
lf = LiveFit([FIT].fit(), y="y_variable", x="x_variable", update_every=0.5)

# Then subscribe to LiveFitPlot(lf, ...)
```

The `[FIT].fit()` function will pass the `FitMethod` object straight to the `LiveFit` class.

**Note:** that for the fits in the above table that require parameters, you will need to pass value(s) to their `.fit` method. For example Polynomial fitting:

```py
lf = LiveFit(Polynomial.fit(3), y="y_variable", x="x_variable", update_every=0.5)
# For a polynomial of degree 3
```

### Option 2: Use custom fits

If you wish, you can define your own non-standard `FitMethod` object. The `FitMethod` class takes two function arguments as follows:

- `model`
- A function representing the behaviour of the model.
- Returns the `y` value (`float`) at the given `x` value and model parameters.
- `guess`
- A function that must take two `np.array` arrays, representing `x` data and respective `y` data, of type `float` as arguments and must return a `dict` in the form `dict[str, lmfit.Parameter]`.
- This will be called to guess the properties of the model, given the data already collected in the Bluesky run.

See the following example on how to define these.

```py
# Linear Fitting

import lmfit

def model(x: float, m: float, c: float) -> float:

return m * x + c # y = mx + c

def guess(x: npt.NDArray[np.float64], y: npt.NDArray[np.float64]) -> dict[str, lmfit.Parameter]:

# Linear regression calculation
# x = set of x data
# y = set of respective y data
# x[n] makes a pair with y[n]

numerator = sum(x * y) - sum(x) * sum(y)
denominator = sum(x**2) - sum(x) ** 2

m = numerator / denominator
c = (sum(y) - m * sum(x)) / len(x)

init_guess = {
"m": lmfit.Parameter("m", m), # gradient
"c": lmfit.Parameter("c", c), # y - intercept
}

return init_guess

fit_method = FitMethod(model, guess)
#Pass the model and guess function to FitMethod

lf = LiveFit(fit_method, y="y_variable", x="x_variable", update_every=0.5)

# Then subscribe to LiveFitPlot(lf, ...)
```

**Note:** that the parameters returned from the guess function must allocate to the arguments to the model function, ignoring the independant variable e.g `x` in this case. Array-like structures are not allowed. See the [lmfit documentation](https://lmfit.github.io/lmfit-py/parameters.html) for more information.

#### Option 2: Continued

Each `Fit` in `ibex_bluesky_core.callbacks.fitting` has a `.model()` and `.guess()`, which make up their fitting method. These are publically accessible class methods.

This means that aslong as the parameters returned from the guess function match to the arguments of the model function, you may mix and match user-made and standard, models and guess functions in the following manner:

```py
import lmfit
from ibex_bluesky_core.callbacks.fitting.fitting_utils import Linear

def different_model(x: float, m: float, c: float) -> float:

return m * x + c ** 2 # y = mx + (c ** 2)


fit_method = FitMethod(different_model, Linear.guess())
# Uses the user defined model and the standard Guessing. function for linear models

lf = LiveFit(fit_method, y="y_variable", x="x_variable", update_every=0.5)

# Then subscribe to LiveFitPlot(lf, ...)
```
... or the other way round ...

```py
import lmfit
from ibex_bluesky_core.callbacks.fitting.fitting_utils import Linear

# This Guessing. function isn't very good because it's return values don't change on the data already collected in the Bluesky run
# It always guesses that the linear function is y = x

def different_guess(x: float, m: float, c: float) -> float:

init_guess = {
"m": lmfit.Parameter("m", 1), # gradient
"c": lmfit.Parameter("c", 0), # y - intercept
}

return init_guess

fit_method = FitMethod(Linear.model(), different_guess)
# Uses the standard linear model and the user defined Guessing. function

lf = LiveFit(fit_method, y="y_variable", x="x_variable", update_every=0.5)

# Then subscribe to LiveFitPlot(lf, ...)
```

Or you can create a completely user-defined fitting method.

**Note:** that for fits that require arguments, you will need to pass values to their respecitive `.model` and `.guess` functions. E.g for `Polynomial` fitting:

```py
fit_method = FitMethod(Polynomial.model(3), different_guess) # If using a custom guess function
lf = LiveFit(fit_method, ...)
```
See the [standard fits](#models) list above for standard fits which require parameters. It gets more complicated if you want to define your own custom model or guess which you want to pass parameters to. You will have to define a function that takes these parameters and returns the model / guess function with the subsituted values.
Binary file added doc/fitting/images_fits/dampedosc.PNG
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added doc/fitting/images_fits/erf.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added doc/fitting/images_fits/erfc.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added doc/fitting/images_fits/gaussian.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added doc/fitting/images_fits/lorentzian.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added doc/fitting/images_fits/slitscan.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added doc/fitting/images_fits/tophat.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added doc/fitting/images_fits/trapezoid.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
143 changes: 143 additions & 0 deletions doc/fitting/standard_fits.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,143 @@
# Standard Fitting Models

## Linear

- `m` - Gradient
- `c` - (y) Intercept

```{math}
y = mx + c
```

## Polynomial

- `a` ... `d` - Polynomial coefficients

For a polynomial degree `n`:
```{math}
y = ax^n + bx^n-1 + ... + cx^1 + d
```

## Gaussian

- `amp` - The maximum height of the Gaussian above `background`
- `sigma` - A scalar for Gaussian width
- `x0` - The centre (x) of the Gaussian
- `background` - The minimum value (y) of the Gaussian

```{math}
y = \text{amp} * e^{-\frac{(x - x0) ^ 2}{2 * \text{sigma}^2}} + \text{background}
```

![GaussianModel](./images_fits/gaussian.png)

## Lorentzian

- `amp` - The maximum height of the Lorentzian above `background`
- `sigma` - A scalar for Lorentzian width
- `center` - The centre (x) of the Lorentzian
- `background` - The minimum value (y) of the Lorentzian

```{math}
y = \frac{\text{amp}}{1 + \frac{x - \text{center}}{\text{sigma}}^2} + \text{background}
```

![LorentzianModel](./images_fits/lorentzian.png)

## Damped Oscillator (DampedOsc)

- `center` - The centre (x) of the oscillation
- `amp` - The maximum height of the curve above 0
- `freq` - The frequency of the oscillation
- `width` - How far away from the centre will oscillations last for

```{math}
y = \text{amp} * \cos((x - \text{center}) * \text{freq}) * e^{-\frac{x - \text{center}}{\text{width}^ 2}}
```

![DampedOscModel](./images_fits/dampedosc.png)

## Slit Scan (SlitScan)

- `background` $b$ - The minimum value (y) of the model
- `inflection0` $i_0$ - The x coord of the first inflection point
- `gradient` $g$ - The gradient of the sloped-linear section of the model
- `inflections_diff` $i_{\Delta}$ - The x displacement between the two inflection points
- `height_above_inflection1` $h_1$ - The y displacement between inflection 1 and the model's asymptote

```{math}
\text{exp_seg} = h_1 \cdot \text{erf} \left( g \cdot \frac{\sqrt{\pi}}{2h_1} \cdot (x - i_0 - \Delta i) \right) + g \cdot \Delta i + b
```

```{math}
\text{lin_seg} = \max(b + g * (x - i_0), b)
```

```{math}
y = \min(\text{lin_seg}, \text{exp_seg})
```

![SlitScanModel](./images_fits/slitscan.png)

## Error Function (ERF)

- `cen` - The centre (x) of the model
- `stretch` - A horizontal stretch factor for the model
- `scale` - A vertical stretch factor for the model
- `background` - The minimum value (y) of the model

```{math}
y = background + scale * erf(stretch * (x - cen))
```

![ERFModel](./images_fits/erf.png)

## Complementary Error Function (ERFC)

- `cen` - The centre (x) of the model
- `stretch` - A horizontal stretch factor for the model
- `scale` - A vertical stretch factor for the model
- `background` - The minimum value (y) of the model

```{math}
y = background + scale * erfc(stretch * (x - cen))
```

![ERFCModel](./images_fits/erfc.png)

## Top Hat (TopHat)

- `cen` - The centre (x) of the model
- `width` - How wide the 'hat' is
- `height` - The maximum height of the model above `background`
- `background` - The minimum value (y) of the model

```{math}
y =
\begin{cases}
\text{background} + \text{height}, & \text{if } |x - \text{cen}| < \frac{\text{width}}{2} \\
\text{background}, & \text{otherwise}
\end{cases}
```

![TopHatModel](./images_fits/tophat.png)

## Trapezoid

- `cen` - The centre (x) of the model
- `gradient` - How steep the edges of the trapezoid are
- `height` - The maximum height of the model above `background`
- `background` - The minimum value (y) of the model
- `y_offset` - Acts as a width factor for the trapezoid. If you extrapolate the sides of the trapezoid until they meet above the top, this value represents the y coord of this point minus height and background.

```{math}
f(x) = \text{y_offset} + \text{height} + \text{background} - \text{gradient} * |x - \text{cen}|
```
```{math}
g(x) = \max(f(x), \text{background})
```
```{math}
y = \min(g(x), \text{background} + \text{height})
```

![TrapezoidModel](./images_fits/trapezoid.png)
8 changes: 7 additions & 1 deletion doc/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -12,14 +12,20 @@ This documentation contains information for the bluesky plan stubs & devices for

devices/*


.. toctree::
:maxdepth: 2
:caption: Callbacks
:glob:

callbacks/*

.. toctree::
:maxdepth: 2
:caption: Fitting
:glob:

fitting/*

.. toctree::
:maxdepth: 2
:caption: Developer information
Expand Down
Loading

0 comments on commit 6cae0a2

Please sign in to comment.