Alternative notebook to convert WFM data to wavelength

[nvaytet]

Supersedes #53.

This is a different approach to computing wavelengths for WFM data.
Instead of runnig the simulation with one opening at a time, finding frame edges (removing outliers), then applying a time offset to each frame, we define a function that connects tof to wavelength.

This is done by plotting wavelength as a function of tof for the neutrons that make it to the detector

(this is the same figure as those made by scippneutron's chopper cascade module).

Then, we take the mean wavelength inside each tof bin, and use that as a 1d linear interpolator that computes wavelengths given an array of tofs

This is 100 times simpler than the old approach and yields very good results:

[jokasimr]

I think we should consider renaming the tof coordinate in this library to toa to avoid frequently having to explain that actually it refers to the time of arrival and not time of flight.

I wanted a figure that more quantitatively illustrates the errors of the two methods.
To do that I made a figure that shows the probability a computed wavelength has relative error above x, as a function of x. I think it came out quite well. Maybe it is something we can include in the notebook.

grid = sc.geomspace('relative_error', 1e-3, 0.2, 100)

err_wfm = sc.abs(events.flatten(to='event').coords['wavelength'] - flat.coords['wavelength']) / events.flatten(to='event').coords['wavelength']
err_naive = sc.abs(events.flatten(to='event').coords['wavelength'] - naive.coords['wavelength']) / events.flatten(to='event').coords['wavelength']

err_wfm = sc.cumsum(err_wfm.hist(relative_error=grid))
err_naive = sc.cumsum(err_naive.hist(relative_error=grid))

p = pp.plot({'wfm': 1 - err_wfm / sc.max(err_wfm), 'naive': 1 - err_naive / sc.max(err_naive)}, scale={'relative_error': 'log'})
p.ax.set_ylabel('Probability of $rel. err. > x$')
p

[nvaytet]

I think we should consider renaming the tof coordinate in this library to toa

I think I agree with this but this would break all the existing notebooks that users have. We should probably keep the old tof coordinate as well for some time period, to avoid too many issues, with a deprecation period?

Shame that the library is called tof... 😬

[nvaytet]

In your calculation of err_wfm, I think some of the wavelengths are NaN (in between frames). Is this properly handled in what you did? (I don't know how cumsum handles NaNs)

[jokasimr]

I think we should consider renaming the tof coordinate in this library to toa

I think I agree with this but this would break all the existing notebooks that users have. We should probably keep the old tof coordinate as well for some time period, to avoid too many issues, with a deprecation period?

Yes that's a problem. But maybe a good opportunity for us to practice doing a proper deprecation cycle.

Shame that the library is called tof... 😬

No I think that's still a fine name!

[jokasimr]

In your calculation of err_wfm, I think some of the wavelengths are NaN (in between frames). Is this properly handled in what you did? (I don't know how cumsum handles NaNs)

That's handled in .hist. Note that we are histogramming a sc.Variable there, so if the variable is nan it doesn't fit in any bin and is dropped.

[nvaytet]

In your calculation of err_wfm, I think some of the wavelengths are NaN (in between frames). Is this properly handled in what you did? (I don't know how cumsum handles NaNs)

That's handled in .hist. Note that we are histogramming a sc.Variable there, so if the variable is nan it doesn't fit in any bin and is dropped.

Even if you look at scipp/scipp#3556 ?

[nvaytet]

If I histogram with enough bins, I can see the nans in there (see left edge in plot). They just don't show up in the other plots because they don't contribute much. This would be fixed by fixing scipp.

[jokasimr]

In your calculation of err_wfm, I think some of the wavelengths are NaN (in between frames). Is this properly handled in what you did? (I don't know how cumsum handles NaNs)

That's handled in .hist. Note that we are histogramming a sc.Variable there, so if the variable is nan it doesn't fit in any bin and is dropped.

Even if you look at scipp/scipp#3556 ?

Good to know! I assumed they were dropped

[jokasimr]

With the toa issue resolved in #59 I'm happy to see this merged