Skip to content

Latest commit

 

History

History
70 lines (63 loc) · 4.4 KB

source_code_structure.md

File metadata and controls

70 lines (63 loc) · 4.4 KB
Raw

GPU Plugin Structure

Historically, GPU plugin was built on top of standalone clDNN library for DNNs inference on Intel® GPUs, but at some point clDNN became a part of OpenVINO, so now it's a part of overall GPU plugin code. Intel® Arc™ Graphics Xe-HPG is supported via embedding of oneDNN library

OpenVINO GPU plugin is responsible for:

  1. IE Plugin API implementation.
  2. Translation of a model from common IE semantic (ov::Function) into plugin-specific one (cldnn::topology), which is then compiled into GPU graph representation (cldnn::network).
  3. Implementation of OpenVINO operation set for Intel® GPU.
  4. Device-specific graph transformations.
  5. Memory allocation and management logic.
  6. Processing of incoming InferRequests, using clDNN objects.
  7. Actual execution on GPU device.

Intel GPU Plugin source code structure is shown below:

src/plugins/intel_gpu                  - root GPU plugin folder
             ├── include
             │   ├── intel_gpu         - library internal headers
             │   │   ├── graph         - headers for internal graph representations
             │   │   ├── plugin        - definition of classes required for OpenVINO plugin API implementation
             │   │   ├── primitives    - primitive definitions for all supported operations
             │   │   └── runtime       - abstraction for execution runtime entities (memory, device, engine, etc)
             │   └── va
             ├── src
             │   ├── graph - all sources related to internal graph representation
             │   │    ├── graph_optimizer - passes for graph transformations
             │   │    ├── impls - definition of primitive implementations
             │   │    └── include - headers with graph nodes
             │   │
             │   ├── kernel_selector - OpenCL™ kernels (host+device parts) + utils for optimal kernels selection
             │   │   ├── common      - definition of some generic classes/structures used in kernel_selector
             │   │   └── core        - kernels, kernel selectors, and kernel parameters definitions
             │   │       ├── actual_kernels  - host side part of OpenCL™ kernels including applicability checks, performance heuristics and Local/Global work-groups description
             │   │       ├── cache  - cache.json - tuning cache of the kernels which is redistributed with the plugin to improve kernels and kernel parameters selection for better performance
             │   │       ├── cl_kernels - templates of GPU kernels (device part) written on OpenCL™
             │   │       └── common - utils for code generation and kernels selection
             │   ├── plugin - implementation of OpenVINO plugin API
             │   │    └── ops - factories for conversion of OpenVINO operations to internal primitives
             │   └── runtime
             │        └── ocl/ - implementation for OpenCL™ based runtime
             ├── tests
             │   ├── test_cases
             │   └── test_utils
             └── thirdparty
                 ├── onednn_gpu - oneDNN submodule which may be used to accelerate some primitives
                 └── rapidjson  - thirdparty RapidJSON lib for reading json files (cache.json)

It is worth it to mention the functional tests, which are located in:

src/tests/functional/plugin/gpu

Most of the tests are reused across plugins, and each plugin only needs to add the test instances with some specific parameters.

Shared tests are located in:

src/tests/functional/plugin/shared                        <--- test definitions
src/tests/functional/plugin/gpu/shared_tests_instances    <--- instances for GPU plugin

See also