Initial support for the Eigen library

.github/workflows/ci.yml

@@ -21,10 +21,10 @@ jobs:
       fail-fast: false
       matrix:
         os: ['ubuntu-latest', 'windows-2022', 'macos-latest']
-        python: ['3.8', '3.9', '3.10', '3.11', '3.12.0-alpha.4', 'pypy-3.9-nightly']
+        python: ['3.8', '3.9', '3.10', '3.11', '3.12.0-alpha.4', 'pypy3.9']
         exclude:
           - os: 'macos-latest'
-            python: 'pypy-3.9-nightly'
+            python: 'pypy3.9'
           - os: 'windows-2022'
             python: '3.12.0-alpha.4'
 
@@ -44,12 +44,16 @@ jobs:
     - name: Update CMake
       uses: jwlawson/[email protected]
 
+    - name: Install Eigen
+      if: matrix.os == 'ubuntu-latest'
+      run: sudo apt-get -y install libeigen3-dev
+
     - name: Install PyTest
       run: |
         python -m pip install pytest pytest-github-actions-annotate-failures
 
     - name: Install NumPy
-      if: matrix.python != 'pypy-3.9-nightly' && matrix.python != '3.12.0-alpha.4'
+      if: matrix.python != 'pypy3.9' && matrix.python != '3.12.0-alpha.4'
       run: |
         python -m pip install numpy
 

docs/changelog.rst

@@ -8,10 +8,39 @@ current version is still in the prototype range (*0.x.y*), there are no (formal)
 guarantees of API or ABI stability. That said, I will do my best to minimize
 inconvenience whenever possible.
 
+Version 0.1.1 (TBA)
+-------------------------------
+* Added casters for dense matrix/array types from the `Eigen library
+  <https://eigen.tuxfamily.org/index.php?title=Main_Page>`_. (PR `#120
+  <https://github.com/wjakob/nanobind/pull/120>`_).
+* Implemented `nb::bind_vector<T>()` analogous to similar functionality in
+  _pybind11_. (commit `f2df8a
+  <https://github.com/wjakob/nanobind/commit/f2df8a90fbfb06ee03a79b0dd85fa0e266efeaa9>`_).
+* Implemented `nb::bind_map<T>()` analogous to similar functionality in
+  _pybind11_. (PR `#114 <https://github.com/wjakob/nanobind/pull/114>`_).
+* Updated tuple/list iterator to satisfy the `std::forward_iterator` concept.
+  (PR `#117 <https://github.com/wjakob/nanobind/pull/117>`_).
+* Fixed issues with non-writeable tensors in NumPy. (commit `cc9cc1
+  <https://github.com/wjakob/nanobind/commit/cc9cc11deb27f8b90bb1b57aaca0f303f87c2d8f>`_).
+* Removed use of some C++20 features from the codebase. This now makes it
+  possible to use _nanobind_ on  Visual Studio 2017 and GCC 7.3.1 (used on RHEL 7).
+  (PR `#115 <https://github.com/wjakob/nanobind/pull/115>`_).
+* Added the `nb::typed<...>` wrapper to override the type signature of an
+  argument in a bound function in the generated docstring. (commit `3404c4
+  <https://github.com/wjakob/nanobind/commit/3404c4f347981bce7f4c7a9bac762656bed8385>`_).
+* Added an `nb::implicit_convertible<A, B>()`` function analogous to the one in
+  _pybind11_. (commit `aba4af
+  <https://github.com/wjakob/nanobind/commit/aba4af06992f14e21e5b7b379e7986e939316da4>`_).
+* Updated `nb::make*_iterator<>` so that it returns references of elements, not
+  copies. (commit `8916f5
+  <https://github.com/wjakob/nanobind/commit/8916f51ad1a25318b5c9fcb07c153f6b72a43bd2>`_).
+* Various minor fixes and improvements.
+* Internals ABI version bump.
+
 Version 0.1.0 (January 3, 2022)
 -------------------------------
 
-* Allow nanobind methods on non-nanobind classes. (PR `#104
+* Allow _nanobind_ methods on non-_nanobind) classes. (PR `#104
   <https://github.com/wjakob/nanobind/pull/104>`_).
 * Fix dangling `tp_members` pointer in type initialization. (PR `#99
   <https://github.com/wjakob/nanobind/pull/99>`_).
@@ -76,7 +105,7 @@ Version 0.0.8 (Oct 27, 2022)
   <https://github.com/wjakob/nanobind/commit/be34b165c6a0bed08e477755644f96759b9ed69a>`_).
 * Caster for ``std::set<..>`` and ``std::unordered_set`` (PR `#87
   <https://github.com/wjakob/nanobind/pull/87>`_).
-* Ported ``nb::make[_key_,_value]_iterator()`` from pybind11. (commit `34d0be1
+* Ported ``nb::make[_key_,_value]_iterator()`` from _pybind11_. (commit `34d0be1
   <https://github.com/wjakob/nanobind/commit/34d0be1bbeb54b8265456fd3a4a50e98f93fe6d4>`_).
 * Caster for untyped ``void *`` pointers. (commit `6455fff
   <https://github.com/wjakob/nanobind/commit/6455fff7be5be2867063ea8138cf10e1d9f3065f>`_).

docs/removed.md

@@ -16,7 +16,7 @@ Removed features include:
 - ○ _nanobind_ instances co-locate instance data with a Python object instead
   of accessing it via a _holder_ type. This is a major difference compared to
   _pybind11_, which has implications on object ownership. Shared/unique
-  pointers are still supported with some restrictions, see below for details.
+  pointers are still supported with some restrictions.
 - ○ Binding does not support C++ classes with overloaded or deleted `operator
   new` / `operator delete`.
 - ○ The ability to run several independent Python interpreters in the same
@@ -28,15 +28,11 @@ Removed features include:
   will not be reintroduced.
 - ○ Module-local types and exceptions are unsupported.
 - ○ Custom metaclasses are unsupported.
-- ● Many STL type caster have not yet been ported.
-- ● PyPy support is gone. (PyPy requires many workaround in _pybind11_ that
-  complicate the its internals. Making PyPy interoperate with _nanobind_ will
-  likely require changes to the PyPy CPython emulation layer.)
 - ◑ NumPy integration was replaced by a more general ``nb::tensor<>``
   integration that supports CPU/GPU tensors produced by various frameworks
   (NumPy, PyTorch, TensorFlow, JAX, ..).
-- ◑ Eigen integration was removed.
-- ◑ Buffer protocol functionality was removed.
+- ◑ Buffer protocol functionality (`.def_buffer()`) was removed, please use the
+  `nb::tensor` interface instead.
 - ◑ Nested exceptions are not supported.
 - ◑ Features to facilitate pickling and unpickling were removed.
 - ◑ Support for embedding the interpreter and evaluating Python code

include/nanobind/eigen.h

@@ -0,0 +1,216 @@
+/*
+    nanobind/eigen.h: type casters for the Eigen library
+
+    The type casters in this header file can pass dense Eigen
+    vectors and matrices
+
+    Copyright (c) 2023 Wenzel Jakob
+
+    All rights reserved. Use of this source code is governed by a
+    BSD-style license that can be found in the LICENSE file.
+*/
+
+#pragma once
+
+#include <nanobind/tensor.h>
+#include <Eigen/Core>
+
+static_assert(EIGEN_VERSION_AT_LEAST(3, 2, 7),
+              "Eigen matrix support in pybind11 requires Eigen >= 3.2.7");
+
+NAMESPACE_BEGIN(NB_NAMESPACE)
+NAMESPACE_BEGIN(detail)
+
+template <typename T>
+using tensor_for_eigen_t = tensor<
+    typename T::Scalar,
+    numpy,
+    std::conditional_t<
+        T::NumDimensions == 1,
+        shape<(size_t) T::SizeAtCompileTime>,
+        shape<(size_t) T::RowsAtCompileTime,
+              (size_t) T::ColsAtCompileTime>>,
+    std::conditional_t<
+        T::IsRowMajor || T::NumDimensions == 1,
+        c_contig, f_contig>
+>;
+
+/// Any kind of Eigen class
+template <typename T> constexpr bool is_eigen_v =
+is_base_of_template_v<T, Eigen::EigenBase>;
+
+/// Detects Eigen::Array, Eigen::Matrix, etc.
+template <typename T> constexpr bool is_eigen_plain_v =
+is_base_of_template_v<T, Eigen::PlainObjectBase>;
+
+/// Detects expression templates
+template <typename T> constexpr bool is_eigen_xpr_v =
+    is_eigen_v<T> && !is_eigen_plain_v<T> &&
+    !std::is_base_of_v<Eigen::MapBase<T, Eigen::ReadOnlyAccessors>, T>;
+
+template <typename T> struct type_caster<T, enable_if_t<is_eigen_plain_v<T>>> {
+    using Scalar = typename T::Scalar;
+    using Tensor = tensor_for_eigen_t<T>;
+    using TensorCaster = make_caster<Tensor>;
+
+    NB_TYPE_CASTER(T, TensorCaster::Name);
+
+    bool from_python(handle src, uint8_t flags, cleanup_list *cleanup) noexcept {
+        TensorCaster caster;
+        if (!caster.from_python(src, flags, cleanup))
+            return false;
+        const Tensor &tensor = caster.value;
+
+        if constexpr (T::NumDimensions == 1) {
+            value.resize(tensor.shape(0));
+            memcpy(value.data(), tensor.data(),
+                   tensor.shape(0) * sizeof(Scalar));
+        } else {
+            value.resize(tensor.shape(0), tensor.shape(1));
+            memcpy(value.data(), tensor.data(),
+                   tensor.shape(0) * tensor.shape(1) * sizeof(Scalar));
+        }
+
+        return true;
+    }
+
+    static handle from_cpp(T &&v, rv_policy policy, cleanup_list *cleanup) noexcept {
+        if (policy == rv_policy::automatic ||
+            policy == rv_policy::automatic_reference)
+            policy = rv_policy::move;
+
+        return from_cpp((const T &) v, policy, cleanup);
+    }
+
+    static handle from_cpp(const T &v, rv_policy policy, cleanup_list *cleanup) noexcept {
+        size_t shape[T::NumDimensions];
+        int64_t strides[T::NumDimensions];
+
+        if constexpr (T::NumDimensions == 1) {
+            shape[0] = v.size();
+            strides[0] = v.innerStride();
+        } else {
+            shape[0] = v.rows();
+            shape[1] = v.cols();
+            strides[0] = v.rowStride();
+            strides[1] = v.colStride();
+        }
+
+        void *ptr = (void *) v.data();
+
+        switch (policy) {
+            case rv_policy::automatic:
+                policy = rv_policy::copy;
+                break;
+
+            case rv_policy::automatic_reference:
+                policy = rv_policy::reference;
+                break;
+
+            case rv_policy::move:
+                // Don't bother moving when the data is static or occupies <1KB
+                if ((T::SizeAtCompileTime != Eigen::Dynamic ||
+                     (size_t) v.size() < (1024 / sizeof(Scalar))))
+                    policy = rv_policy::copy;
+                break;
+
+            default: // leave policy unchanged
+                break;
+        }
+
+        object owner;
+        if (policy == rv_policy::move) {
+            T *temp = new T(std::move(v));
+            owner = capsule(temp, [](void *p) noexcept { delete (T *) p; });
+            ptr = temp->data();
+        }
+
+        rv_policy tensor_rv_policy =
+            policy == rv_policy::move ? rv_policy::reference : policy;
+
+        object o = steal(TensorCaster::from_cpp(
+            Tensor(ptr, T::NumDimensions, shape, owner, strides),
+            tensor_rv_policy, cleanup));
+
+        return o.release();
+    }
+};
+
+/// Caster for Eigen expression templates
+template <typename T> struct type_caster<T, enable_if_t<is_eigen_xpr_v<T>>> {
+    using Array = Eigen::Array<typename T::Scalar, T::RowsAtCompileTime,
+                               T::ColsAtCompileTime>;
+    using Caster = make_caster<Array>;
+    static constexpr auto Name = Caster::Name;
+    template <typename T_> using Cast = T;
+
+    /// Generating an expression template from a Python object is, of course, not possible
+    bool from_python(handle src, uint8_t flags, cleanup_list *cleanup) noexcept = delete;
+
+    template <typename T2>
+    static handle from_cpp(T2 &&v, rv_policy policy, cleanup_list *cleanup) noexcept {
+        return Caster::from_cpp(std::forward<T2>(v), policy, cleanup);
+    }
+};
+
+/// Caster for Eigen::Map<T>
+template <typename T, int MapOptions, typename StrideType>
+struct type_caster<Eigen::Map<T, MapOptions, StrideType>> {
+    using Map = Eigen::Map<T, MapOptions, StrideType>;
+    using Tensor = tensor_for_eigen_t<Map>;
+    using TensorCaster = type_caster<Tensor>;
+    static constexpr auto Name = TensorCaster::Name;
+    template <typename T_> using Cast = Map;
+
+    TensorCaster caster;
+
+    bool from_python(handle src, uint8_t flags,
+                     cleanup_list *cleanup) noexcept {
+        return caster.from_python(src, flags, cleanup);
+    }
+
+    static handle from_cpp(const Map &v, rv_policy, cleanup_list *cleanup) noexcept {
+        size_t shape[T::NumDimensions];
+        int64_t strides[T::NumDimensions];
+
+        if constexpr (T::NumDimensions == 1) {
+            shape[0] = v.size();
+            strides[0] = v.innerStride();
+        } else {
+            shape[0] = v.rows();
+            shape[1] = v.cols();
+            strides[0] = v.rowStride();
+            strides[1] = v.colStride();
+        }
+
+        return TensorCaster::from_cpp(
+            Tensor((void *) v.data(), T::NumDimensions, shape, handle(), strides),
+            rv_policy::reference, cleanup);
+    }
+
+    operator Map() {
+        Tensor &t = caster.value;
+        return Map(t.data(), t.shape(0), t.shape(1));
+    }
+};
+
+/// Caster for Eigen::Ref<T>
+template <typename T> struct type_caster<Eigen::Ref<T>> {
+    using Ref = Eigen::Ref<T>;
+    using Map = Eigen::Map<T>;
+    using MapCaster = make_caster<Map>;
+    static constexpr auto Name = MapCaster::Name;
+    template <typename T_> using Cast = Ref;
+
+    MapCaster caster;
+
+    bool from_python(handle src, uint8_t flags,
+                     cleanup_list *cleanup) noexcept {
+        return caster.from_python(src, flags, cleanup);
+    }
+
+    operator Ref() { return Ref(caster.operator Map()); }
+};
+
+NAMESPACE_END(detail)
+NAMESPACE_END(NB_NAMESPACE)

include/nanobind/nb_lib.h

@@ -371,7 +371,8 @@ NB_CORE dlpack::tensor *tensor_inc_ref(tensor_handle *) noexcept;
 NB_CORE void tensor_dec_ref(tensor_handle *) noexcept;
 
 /// Wrap a tensor_handle* into a PyCapsule
-NB_CORE PyObject *tensor_wrap(tensor_handle *, int framework) noexcept;
+NB_CORE PyObject *tensor_wrap(tensor_handle *, int framework,
+                              rv_policy policy) noexcept;
 
 // ========================================================================
 

include/nanobind/nb_traits.h

@@ -144,4 +144,14 @@ constexpr bool is_detected_v = detail::detector<void, Op, Arg>::value;
 template <typename T>
 using remove_opt_mono_t = typename detail::remove_opt_mono<T>::type;
 
+template <template <typename> typename Base, typename T>
+std::true_type is_base_of_template(const Base<T>*);
+
+template <template <typename> typename Base>
+std::false_type is_base_of_template(...);
+
+template <typename T, template <typename> typename Base>
+constexpr bool is_base_of_template_v =
+    decltype(is_base_of_template<Base>(std::declval<T *>()))::value;
+
 NAMESPACE_END(NB_NAMESPACE)

include/nanobind/nb_types.h

@@ -157,7 +157,7 @@ class handle : public detail::api<handle> {
     NB_INLINE handle(const PyTypeObject *ptr) : m_ptr((PyObject *) ptr) { }
 
     const handle& inc_ref() const & noexcept {
-#if defined(NDEBUG)
+#if defined(NDEBUG) && !defined(Py_LIMITED_API)
         Py_XINCREF(m_ptr);
 #else
         detail::incref_checked(m_ptr);
@@ -166,7 +166,7 @@ class handle : public detail::api<handle> {
     }
 
     const handle& dec_ref() const & noexcept {
-#if defined(NDEBUG)
+#if defined(NDEBUG) && !defined(Py_LIMITED_API)
         Py_XDECREF(m_ptr);
 #else
         detail::decref_checked(m_ptr);

include/nanobind/tensor.h

@@ -328,9 +328,9 @@ template <typename... Args> struct type_caster<tensor<Args...>> {
         return value.is_valid();
     }
 
-    static handle from_cpp(const tensor<Args...> &tensor, rv_policy,
+    static handle from_cpp(const tensor<Args...> &tensor, rv_policy policy,
                            cleanup_list *) noexcept {
-        return tensor_wrap(tensor.handle(), int(Value::Info::framework));
+        return tensor_wrap(tensor.handle(), int(Value::Info::framework), policy);
     }
 };