Extend the usage of pcv.transform.warp

[DannieSheng]

Currently the pcv.transform.warp function can only (perspectively) warp a grayscale or binary image, I propose that we make it also capable of warping an rgb image.

Also, this warping function uses perspective transformation in opencv, I think we can also include affine transformation.

[dschneiderch]

1. yes, it would be great to apply to RGB images. I tried to do this but was running into issues and ran out of steam to adapt for all image types. My use case was specifically for grayscale NIR images or binary masks. I would love to see this adapted for all image types.

2. I had an affine version too, but perspective gives better results for the transformation. the downside is that perspective requires 4 pts instead 3. I didn't think this was a big deal. based on other discussion we should provide both.

[DannieSheng]

I agree. I am working on testing using also RGB images.

[DannieSheng]

When testing on this function, I came across another issue which has occurred several times:
For most openCV functions I've played with, uint8 datatype is required for input image arrays. For myself I have made a function to do this, which is based on the plantCV function "pcv.transform.rescale", which only works for grayscale images. So if there is no equivalent in plantcv, I propose also include this function to plantcv, or simply extend current pcv.transform.rescale to multi-band cases.

[nfahlgren]

Since this is changing data types, rather than rescaling, would it make sense to have a data type converter that maybe could use the scikit-image img_as_* methods? https://scikit-image.org/docs/dev/api/skimage.html?highlight=float#utility-functions

[DannieSheng]

That makes sense, thank you!

[DannieSheng]

An update, the scikit-image img_as_* functions would only accept images whose values between -1 and 1, which would not be true for thermal images. So potentially extra check would be needed.

[nfahlgren]

Oh right, the thermal data wouldn't be in normal image numeric ranges. Maybe for the short term pcv.transform.warp could still take in two regular images and thermal data could be handled like this:

aligned_img = pcv.transform.warp(img=img, refimg=pcv.transform.rescale(gray_img=therm), pts=pts, refpts=refpts)

Later, we could store a scaled image automatically with the thermal data in a class like we do with hyperspectral data and the accompanying RGB image we use as a previewer.

[DannieSheng]

I agree. So for the warp function we would require (assume) that input data are in normal image numeric ranges, and users are responsible for converting irregular data to accepted ones. The idea of storing scaled image automatically with the thermal data in a class sounds like a good idea!

[DannieSheng]

Addressed in #720

Updates for the warp function

1. Return warped image and the geometric transformation matrix
2. Handles not only grayscale / binary images
3. When visualize target image and reference image with markers plotted on them, show inliers and outliers with different markers (only applicable for cases some robust estimation methods are adopted)
4. The datatype of warped image is kept the same as the input image

Added another function warp_align, which takes a target image, a transformation matrix, and an optional reference image as inputs, warp the target image. For visualization, if reference image is provided, the blended image of reference image and warped image would be shown; otherwise, the blended image of warped image and original image would be shown.

[DannieSheng]

Because the openCV function cv2.warpPerspective won't take input image with datatype int, so when input image is a binary image whose datatype is int, it won't work. Either convert the datatype to float or use pcv.transform.rescale would work. I did not include this "check and covert" part inside the function.