Using OpenCL and DLPrimitives from Swift

[philipturner]

@artyom-beilis - citing #13 (reply in thread), this is the thread where we can discuss stuff besides debugging or optimizing DLPrimitives for the Apple architecture.

I plan to take this path:

1. Make the first idiomatic Swift bindings for OpenCL, functioning similarly to opencl.hpp. This is a work in progress: SwiftOpenCL
2. Translate the tests in OpenCL-CLHPP from C++ to my new Swift API, using them as package tests
3. Use my Swift OpenCL bindings as the foundation for the Swift for TensorFlow (S4TF) OpenCL backend. Instead of depending on the DLPrimitives repository or translating its entire code base to Swift, I will just copy the GPU kernels and replicate how your code executes them. I will give appropriate attribution to you and link back to your repository.

Questions:

• How relevant is my SwiftOpenCL package to our conversation?
• Would it be better if I asked the maintainers of OpenCL-CLHPP to periodically compile my Swift bindings and run the tests?
• Is it worth it for you to get experience downloading and using a Swift toolchain now, given that it could be several months before I embark on the DLPrimitives-based backend for S4TF?
• How frequently do you expect to commit to your repository in the future? I will need to periodically check on your repository to incorporate your most recent changes.

[philipturner]

I compiled SwiftOpenCL on Google Colab (which has an Nvidia GPU) and I just ran clGetPlatformIDs and clGetDeviceIDs, which both return 0 😄. If you want to test it out on Swift-Colab, it is much faster than waiting to download a Swift toolchain onto your local computer. I haven't properly documented the new release of Swift-Colab, which allowed me to compile SwiftOpenCL effortlessly, but I can show you how to use it.

[artyom-beilis]

I strongly discourage you from rewriting c++ code in Swift.

1. Project isn't 100% stable and you'll see many modifications and bug fixes.
2. Most of the dlprimitives code is actually c++. Kernels are only small parts of the code
3. The public API is rather small so it is way more logical to call public core API from swift rather than reimplementing all from the scratch

Most of API you need is here http://dlprimitives.org/docs/namespacedlprim_1_1core.html + several classes like Tensor Context etc.

Once you wrap them in Swift you are going to simply maintenance and can work on your own developments

Now you have two approaches:

1. Wrap c++ with some kind of tool for Swift interoperability - like Swig (ajso I'm not familiar with swift to tell what is best toolkit)

2. Add C API to DlPrimitives which can be useful for other projects as well since it simplifies bindings and than call C directly from Swift

   With this i can probably help. I did this for several other C++ projects to provide C interop layer for calling from Java/python/C#.

But implementing it all from the scratch is waste if time.

BTW Swift opencl binding can indeed be very useful

[philipturner]

How much access does DLPrimitives give me to the low-level components of stuff (e.g. can I allocate the buffer’s memory elsewhere and pass it into a tensor)? I will need to make several other trivial kernels in OpenCL, which are not from DLPrimitives. I could bind directly to the DLPrimitives code for MMX kernels if there is enough flexibility.

Binding from Swift to C++ is no problem. It only takes a bit of magic with Clang modules. Also, I bound Swift to the LLDB C++ API to make the new Swift-Colab.

Whatever course this takes, making Swift OpenCL bindings will increase my familiarity with OpenCL.

[artyom-beilis]

the low-level components of stuff (e.g. can I allocate the buffer’s memory elsewhere and pass it into a tensor

Yes, of course. In fact in pytorch dlprimitivies/opencl backend the memory management is separate from dlprimitives

Edit: this is an example of creating dlprimitives tensort from pytorch opencl tensor: https://github.com/artyom-beilis/pytorch_dlprim/blob/master/src/utils.cpp#L43

I will need to make several other trivial kernels in OpenCL, which are not from DLPrimitives

1st of all there are pointwise operations with broadcasting/reduction for generic operations. I use them extensively in pytorch.

For example: https://github.com/artyom-beilis/pytorch_dlprim/blob/master/src/pointwise_ops.cpp#L477 this is how I use it from pytorch.

However there is no direct support of running your own full kernel (it is related to caching) but shouldn't be a big problem to add such stuff. It maybe useful mostly because dlprimitives can cache compiled kernels on disk/memory

[philipturner]

I am planning to keep built-in support for OpenCL versions earlier than 1.2, which is part of opencl.cpp. Is that a good idea?

[artyom-beilis]

I am planning to keep built-in support for OpenCL versions earlier than 1.2, which is part of opencl.cpp. Is that a good idea?

In general dlprimitives even running on Mesa OpenCL 1.1 driver for AMD. But 1.2 is generally more than enough and 1.2 is most widely supported version.

opencl.cpp - what are you talking about AFAIR opencl sdk includes only headers...

[philipturner]

I meant opencl.hpp. It's the header in this repository. It's 9000 lines, but mostly boilerplate. I have to translate that behemoth into Swift to complete this project! But once I get the hang of it, most of the thousands of lines should go quickly.

[artyom-beilis]

I meant opencl.hpp. It's the header in this repository. It's 9000 lines, but mostly boilerplate. I have to translate that behemoth into Swift to complete this project! But once I get the hang of it, most of the thousands of lines should go quickly.

Basically it is C++ wrapper over C API that does may nice C++ tricks with memory management, ref-counting constants etc.

I don't know how it works in Swift but C++ loves what is called RAII and it wraps C OpenCL objects (it is object oriented API) with constructors destructors and nice C++ ideoms and syntactic C++ sugar like device().getInfo<CL_DEVICE_VENDOR_ID>(). I'm not 100% sure if you need to mimic what C++ wrapper does because it is very "C++"

[philipturner]

I'm mimicing C++ wrappers with Swift reference counting. Instead of RAII, Swift uses ARC which is thread-safe. To count a reference, they increment an atomic counter on a pointer. I plan to align OpenCL's C-level reference counting with ARC:

public class CLDevice { // Managed by ARC
  public var deviceID: Int
  private var retainObject: Bool
  public init(deviceID: Int, retainObject: Bool = false) {
    if (/* OpenCL driver >= 1.2.0 */ && retainObject) {
      clRetainDevice(deviceID)
    }
    self.deviceID = deviceID
    self.retainObject = retainObject
  }
  deinit { // Called when ARC deallocates the object
    if (/* OpenCL driver >= 1.2.0 */ && retainObject) {
      clReleaseDevice(deviceID)
    }
  }
}

Swift is known for an extreme amount of syntactic sugar. I'm going to map template functions like device().getInfo<CL_DEVICE_VENDOR_ID>() to either of the following:

let _ = CLDevice().getInfo(.vendorID)
let _ = CLDevice().vendorID

[philipturner]

I've decided to go with translating DLPrimitives from C++ to Swift, and I have some questions:

1. Do you have any advice, such as external dependencies that I should consider?
2. What were the biggest bugs or obstacles you encountered while creating DLPrimitives, and how did you solve them?