[ENH] 4+ Angle Polarisation widget and Hyper Spectra vector plot

[callumgassner]

This is a continuation of #615

Introduces a widget ("4+ Angle Polarisation") to calculate molecular orientation information from polarised vibrational spectroscopy methods collected at 4 or more polarisation angles.
Adds features to hyperspectra which compliments results from 4+ angle polarisation widget:

• Added ability to overlay a vector field on hyperspectral images
• Added "Jet", "Viridis" and "HSV" matplotlib colour maps

The 4+ Angle Polarisation (polar) widget can be manually tested with the datasets in the PR (the 4 files named 4-angle-ftir_multiin1.tab, ... 4.tab).
As outlined in the documentation, these datasets can be loaded into quasar and passed to the polar widget 1 at a time with the "File" widget or all at once with the "Multifile" widget.
When using the former, connect the file widget which loaded the file 4-angle-ftir_multiin1.tab to the polar widget first, then 4-angle-ftir_multiin2.tab, and so on. Using this method, as you connect the file widgets a list should be constructed within the "inputs" section of the polar widgets GUI showing the connected widgets name and a box below each name to enter in the polarisation angle at which this data was collected. The polarisation angles will be entered automatically with a spacing of 180/# of inputs.
Alternatively, using the multifile widget to load the data, when the multifile widget is connected to the polar widget the drop down menu in the Inputs section of the polar widget becomes enabled. This allows the data to be sorted into the individual polarisation angles by any categorical variable - in this instance the Filename variable. As with the previous method, once the filename variable is selected from the menu a list is created, this time with each category as a separate polarisation angle.

For testing the resulting data from this widget, within the features column select the features 1214.0 - 1258.0 and 1500.0 - 1580.0, set the X and Y axis to map_x and map_y, respectively. The alpha value should be 0 and the invert angles checkbox should be selected. Click apply and connect the "Polar Data" output of the polar widget to a hyperspectral widget.
Within the hyperspectral widget, use the "viridis" colourmap and plot the colourmap with either "Intensity" feature. To plot the vector plot, select the "plot vector overlay" checkbox, use the Azimuth Angle as the Vector angle and the Amplitude as the Vector magnitude - the scale slider can be used to reduce or increase the respective lengths of the vectors if they are too long. Results should be almost identical to the below image:

The majority of changes since then have been bug fixes. Although there have been some changes to the widget's themselves:

4+ Angle Polarisation:

• Added the "Spectra Type" parameter: If the input is not in absorptance (α = 1-T), the spectra are first converted to it
• Changed "Alpha value" to "TDM Tilt": Avoids confusion between the alpha in the referenced paper (Hikima et al. (2013)) and absorptance (α is also most commonly used to refer to absorptance)
• Included and option to average the spectra across the polarisation angles so that the meta/class variables are not repeated 4x (or more) times.

Hyperspectra:

• Move vector overlay checkbox to Image values box, and only show vector controls when the overlay is enabled
• Width slider to fatten up the vectors when needed
• Ability to colour each vector depending on a third parameter and mapped to a separate colourmap
• The above introduces a second colourbar to the hyperspectra image, so added the ability to label each colourbar
• Added ability to bin pixels for vectors only: often the vectors are illegible for large datasets, the ability to average the values within a x by x square makes this much clearer

These changes have all been implemented based on feedback from the previous pr and initial users.

I believe many of the issues raised in the original pr have been addressed to some extent. However, I am aware of a few possible issues the persist:

Hyperspectra:

• RuntimeWarning is raised when np.nanmean encounters an empty array. This can either be caught and silenced, or the arrays can be checked first (which is much slower)

4+ Angle Polarisation:

• I feel that there might still some issues with ContextSettings. Under some conditions on new installs/widget setting resets the settings persist for incompatible datasets, and so errors are raised. I have been dealing with them as they appear and I'm not aware of any specific ones at the moment, but if you come across any please let me know.

• There is still a possible memory leak (small). Memory usage creeps up by small increments for repeated (complete and interrupted) runs of the widget. I am not sure if this is specific to this widget as I've seen similar behaviour with e.g. peak fit.

• For me the documentation of this widget does not show

• It would be better to enable users to enter the angle of the TDM tilt per vibrational mode. As was pointed out in the previous pr, this would be unique to each one (if the specific angle is known, often it is approximated as either parallel (0 degrees) or perpendicular (90 degrees)). But I can not think of any elegant way to implement this - more than happy to take suggestions on this.

• I will need to also update the documentation for this widget prior to it being merged.

I just thought it would be better to get the ball rolling for now (and see what possibly gets purged - I acknowledge that some of these additions are niche and add significant complexity to the code, so I understand if some things do not get merged in the final version :) ).

Final note - Based on the combine_visimg function in owpolar, I have a very similar function for the multifile widget for stacking visible images from multiple files. If you are interested I can create a new pr or add the changes to this one (there was brief mention of this before activity on the other pr fizzled out)

[markotoplak]

Hyperspectra:

* RuntimeWarning is raised when np.nanmean encounters an empty array. This can either be caught and silenced, or the arrays can be checked first (which is much slower)

Ignore them. Silencing them is not appropriate for multithreaded applications (those impose global states). Numpy 2 to the rescue, some day.

[markotoplak]

• For me the documentation of this widget does not show

If I build the docs and install as a pip package, docs show up as expected. Thanks!

[markotoplak]

My last commit changed the UI a bit. My goal was to unify visual image and vector plot settings. Then I also made vector plot settings more compact.

Before:

After:

[markotoplak]

To me, the addition to HyperSpectra seems good. I asked some questions where I was unsure, but none are blockers.

What I'd like is to have it refactored in a Mixin-type class that will attach to ImagePlot (as too many already do). That will help to keep all related functionality (with settings and public vars) in one class that needs minimal interaction outside of it. Also, changes to the ImagePlot class should be kept at the minimum (if something is needed, a call to the appropriate Mixin function should be made.

@callumgassner, please decide if I do this or you. I saw that most of the code concerning vectors is functionally well separated so this is mostly about moving methods and changing self.imageplot to self (and probably adapting tests slightly).

Also, even if the interface shows up in the parent (control area) it can be totally defined in a Mixin - see how I moved the axis and color selectors in #777.

[callumgassner]

Hi Marko,

I am not sure if my last commit is what you meant by your last comment, but I was curious to give it a shot myself. I think the readability is much better like this, so maybe I am not to far off?

[markotoplak]

@callumgassner, this is exactly what I had in mind, thanks!

The only thing I would add would be to move them from the BasicImagePlot into the next class, because BasicImagePlot is (not so smartly :) ) used in another add-on, Orange-SNOM, and needs to work without vector plots.

The tests also fail not but should be trivial to fix.

[markotoplak]

@callumgassner, what do you think of the proposed HyperSpectra UI reorganization?

[markotoplak]

The only thing I would add would be to move them from the BasicImagePlot into the next class, because BasicImagePlot is (not so smartly :) ) used in another add-on, Orange-SNOM, and needs to work without vector plots.

The tests also fail not but should be trivial to fix.

I did what I suggested and removed (almost) everything vector-plot-related from the BasicImagePlot. The Preprocess Images widget from SNOM now seems to work just fine. Please ensure that I did not break anything.

[markotoplak]

• I feel that there might still some issues with ContextSettings. Under some conditions on new installs/widget setting resets the settings persist for incompatible datasets, and so errors are raised. I have been dealing with them as they appear and I'm not aware of any specific ones at the moment, but if you come across any please let me know.

You have a sharp eye for issues. You saw the following bug: biolab/orange3/issues/6721. So the solution is to use "exclude_attributes=True" and "exclude_classes=True" for settings that can only be metas (because the corresponding domain model is defined as such), such as was done in #707.

EDIT: I fixed this.

[markotoplak]

• It would be better to enable users to enter the angle of the TDM tilt per vibrational mode. As was pointed out in the previous pr, this would be unique to each one (if the specific angle is known, often it is approximated as either parallel (0 degrees) or perpendicular (90 degrees)). But I can not think of any elegant way to implement this - more than happy to take suggestions on this.

Why not make a new box for entering a number with a sensible default? I have a feeling I do not understand this, please explain what you meant. Thanks!

[markotoplak]

• There is still a possible memory leak (small). Memory usage creeps up by small increments for repeated (complete and interrupted) runs of the widget. I am not sure if this is specific to this widget as I've seen similar behaviour with e.g. peak fit.

Let's not worry about this now.

[markotoplak]

@callumgassner, I rebased to current master add added some slight changes (see last commits).

I think this is almost good to go. We only need:

• Updated docs.
• Info on citing. I suggest just making a box with a label in the Polar widget. I think this is what we decided on the last meeting.

Note that in the refactor in HyperSpectra all vector settings were moved into ImagePlot. Thus, old workflows will loose HyperSpectra settings and they should be recreated (probably easier) or a migration should be written (harder, I would not do it unless you really need it).

[callumgassner]

Thanks for the help, Marko!
I've updated the doc and added the available citation information - there may be some additional ones that should be added but I will need to confirm this.

@callumgassner, what do you think of the proposed HyperSpectra UI reorganization?

I think this is better, important controls are more easily accessible now.

• It would be better to enable users to enter the angle of the TDM tilt per vibrational mode. As was pointed out in the previous pr, this would be unique to each one (if the specific angle is known, often it is approximated as either parallel (0 degrees) or perpendicular (90 degrees)). But I can not think of any elegant way to implement this - more than happy to take suggestions on this.

Why not make a new box for entering a number with a sensible default? I have a feeling I do not understand this, please explain what you meant. Thanks!

Basically, we would need a new field for each wavenumber value, so something like a linedit box next to each unique value in the "features" column. I briefly played around with this today using a QTableView widget but I wasn't very happy with how it was looking:

[markotoplak]

Basically, we would need a new field for each wavenumber value, so something like a linedit box next to each unique value in the "features" column. I briefly played around with this today using a QTableView widget but I wasn't very happy with how it was looking

Ah, now I get it. If it is per-wavenumber, then, well, what you prototyped makes sense in terms that you have to show both to the user. But then you have to combine settings with selections... I would do it in similarly as in the Discretize widget. Check it out!

[markotoplak]

@callumgassner, I added a commit than changes Polar layout a bit. I removed boxes in boxes (which we try not to use), slightly reworded labels, and moved citations. Feel free to remove/edit the commit - I do not have a strong preference.

[callumgassner]

Basically, we would need a new field for each wavenumber value, so something like a linedit box next to each unique value in the "features" column. I briefly played around with this today using a QTableView widget but I wasn't very happy with how it was looking

Ah, now I get it. If it is per-wavenumber, then, well, what you prototyped makes sense in terms that you have to show both to the user. But then you have to combine settings with selections... I would do it in similarly as in the Discretize widget. Check it out!

I quite liked this idea, thanks!! I've used it to add the unique values per feature.
I am also happy with the changes to the layout :)

[markotoplak]

I did not test your new feature but I trust you did. So, do we merge?

[callumgassner]

I am happy for it to be merged :)