Slides:
Attendees:
- Svetlana Podogova (Intel)
- Victor Getmanskiy (Intel)
- Mark Rabotnikov (Philips)
- Ashish Uthama (Mathworks)
- Tim van der Horst (Philips)
- Robert Schneider (Siemens Healthineers)
- Sergey Ivanov (Intel, OpenCV/G-API)
- Maksim Shabunin (Intel, OpenCV)
- Sohrab Amirghodsi (Adobe)
- Valentin Kubarev (Intel)
- Aleksandr Duev (Intel)
- Dmitriy Budnikov (Intel)
- Alison L Richards (Intel)
Agenda:
- Introduction and Open questions discussion
- Hardware-accelerated images and data formats
- oneIPL image data abstraction
- oneIPL image interoperability with USM
- Closing words and next plans on oneIPL TAB
Open question discussion:
Problem statement: Accuracy across devices is different due to CPU and GPU devices support different IEEE754 compliance, the standard libraries has no claims on correct rounding and the order of operations impacts the result since the algorithms has different flows on different devices.
oneIPL suppose to control the precision of computations withing supported computation datatype – ComputeT, which is a template parameter of the functions. By default, it is float. Double (if it is supported by device) could be more precise but slow, and half might be faster but with low accuracy.
- To discuss:
- Are there any specific expectations from image processing perspective?
- How important is the accuracy for different use-cases?
- Are there any criteria on the results similarity across devices?
- Are there any image similarity metrics not related to accuracy, which required to be fulfilled like PSNR?
Ashish Uthama, Mathworks: For CPU and GPU comparison we see this difference and can tolerate +-1 pixel variation. For the floating point it depends on the pipeline depth: how many calculations you have.
Requirement: it should be ok for the whole pipeline, no other special expectations. But it is important to have reproducibility - to have the same result on the same device. Across different devices difference in pixels is ok.
Another thing is that was observed by Mathworks is different run-to-run results while using threading – it is important to have similar run-to-run results.
Mathworks use pixel-wise comparison, and sometimes they use structural similarity metrics for regression tests. Ashish Uthama shared the link for structural similarity article.
Tim van der Horst, Philips: the same, we also do pixel wise comparison.
Robert Schneider, Siemens Healthineers: Accuracy not necessarily need to be the same across CPU and GPU, but it should not be too great difference, should not add artifacts or noises.
+-1 is ok, but also there should not be big difference according to visual feeling, and AI algorithms should not be messed by this difference.
oneIPL specification walk-through:
The latest version of oneIPL Spec is published on oneIPL Spec page.
Current provisional spec version is 0.5, the spec 0.6 is in progress.
- The main difference between spec ver 0.5 and ver 0.6 is the following:
- ipl::formats replaced by ipl::layouts
- Image constructors are changed to remove dependency on implementation
- Default image allocator will be USM
- Methods to image auxiliary classes moved to image API
- Switched to generic template parameters
- Gaussian filter with separated sigma for x and y axis
- Normalize without sycl::buffer in spec
sycl::image supported formats list was reduced. Mostly only the C4 ipl::image formats are mapped to hardware-accelerated images.
Currently there is no support for SYCL 2020 images in compilers, so hardware images are used only inside implementation and there is no option to have any public API working with SYCL images in spec.
SYCL image still is incomplete. SYCL 2020 image still has no implementation in compilers. So public API has no option to use sycl::image, the option is only the implementation and image(texture) memory access.
The list of supported formats was reduced in SYCL 2020. Q: Is there any feedback on this change?
Robert Schneider, Siemens Healthineers: grayscale images are very important and the texture images are also blocking.
Ashish Uthama, Mathworks: agree.
Tim van der Horst, Philips: agree, grayscale image format is very important for medical.
Mark Rabotnikov, Philips (offline feedback): Support of gray-scale images: all medical scanners (like CT, MR, PET, etc.) create gray-scale images, usually 16-bit, but can be also 8-bit. In addition, gray-scale 32-bit floating point images are widely used during processing, including AI, etc. So in my view it is very important to fully support such images.
One of the important changes in oneIPL Spec v0.6 is changing from Formats to Layouts. Reasons:
- Layout is a part of compile-time dispatch and defines algorithm in kernel, multiple formats are mapped to single layout: channel4 layout -> rgba/bgra/argb/abgr/cmyk/… formats
- Align to industry standard approach (OpenCV, python libraries, Intel® Integrated Performance Primitives/NVIDIA Performance Primitives.
Significantly affected functions: Color conversions, which requires specific color format.
Ashish Uthama, Mathworks: Does the scope of oneIPL supported formats include 3D data or only 2D? Victor Getmanskiy, Intel: The only 2D formats are included right now, but it would be extended in future to cover 3D also (Reference: slide 5 in oneIPL TAB #1 presentation )
Basic terminology is discussed (Region of Interest, pitch, width, length)
oneapi::ipl::image class is basic data abstraction for image data in oneIPL. oneIPL provides single abstraction over different memory types: host, device, shared and special GPU images (textures).
The ipl::image class is reviewed, API are discussed:
Robert Schneider, Siemens Healthineers: What is returned by get_pointer() function?
Victor Getmanskiy, Intel: this is the pointer to the full image
Robert Schneider, Siemens Healthineers: and what if the hardware texture is used?
Victor Getmanskiy, Intel: this is very important question. It can be extended as soon as the texture images are added to the SYCL standard with capability to return device memory. But now it is hard to introduce it in the spec, since it is not allowed in SYCL standard.
Robert Schneider, Siemens Healthineers: It would be good to make API more forward looking, more general to take into account future potential extension. For example, introduce some structure / class / handler for it.
Ashish Uthama, Mathworks: we may use more specific naming for this function, something like get_USM_pointer().
Agreed for all TAB members to take offline review of slide 14 of oneIPL TAB #2 presentation and provide the suggestion for the more general API ideas and the function naming.
Mark Rabotnikov, Philips (offline feedback): Regarding the naming of the methods in image class: in my view there is some inconsistency in naming with respect to ROI. For example, I think it would be clearer to change get_width/get_height to get_roi_width/get_roi_height.
Sergey Ivanov, OpenCV/G-API (offline feedback): I believe, the is_roi(), get_roi_rect(), get_full_image(), get_roi(), get_width(), get_height(), get_full_width() and get_full_height() functions are redundant in image class This can be represented by 3 methods only:
- const roi_rect& get_rect() const;
- auto original() const ->image;
- auto apply_roi(const roi_rect& roi_rect) const ->image;
- Then, there will be the following:
- object.is_roi() -> object.get_rect() == object.original().get_rect()
- object.get_full_width() -> object.original().get_rect().width()
- object.get_full_height() -> object.original().get_rect().height()
- object.get_width() -> object.get_rect().width()
- object.get_height() -> object.get_rect().height()
Also, I suppose, that get_pointer() should be part of USM template specialization only, instead it is possible to introduce:
- auto get_handle() -> cl::sycl::handle
The different image constructors are discussed.
The example of custom kernel with the USM usage is discussed.
See more details on oneIPL Image Data Abstraction page
Next plans on oneIPL TAB:
- The next technical meeting for oneIPL TAB is planned for February 17th (ww8)
- Next topic for the discussion is Memory Allocation and oneIPL Library design details.