Skip to content

Commit

Permalink
publish first version of partio to PyPi
Browse files Browse the repository at this point in the history
  • Loading branch information
alexus37Test committed Jul 7, 2021
1 parent e34c2c6 commit 74ad1ff
Show file tree
Hide file tree
Showing 7 changed files with 176 additions and 66 deletions.
3 changes: 3 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -24,3 +24,6 @@
workspace.mel
.vscode/
build_partio/
dist/
partio.egg-info
wheelhouse/
6 changes: 5 additions & 1 deletion CMakeLists.txt
Original file line number Diff line number Diff line change
Expand Up @@ -61,9 +61,13 @@ set(CMAKE_CXX_STANDARD_REQUIRED ON)
set(CMAKE_CXX_EXTENSIONS OFF)
set(CMAKE_FIND_PACKAGE_RESOLVE_SYMLINKS ON)
set(CMAKE_INSTALL_MESSAGE LAZY)
set(CMAKE_SKIP_RPATH ON)
set(CMAKE_THREAD_PREFER_PTHREAD ON)
set(THREADS_PREFER_PTHREAD_FLAG ON)
# this is use for building the python wheel
set(CMAKE_SKIP_RPATH OFF)
set(CMAKE_BUILD_RPATH "$ORIGIN")
set(CMAKE_INSTALL_RPATH "$ORIGIN")


## Setup platform specific helper defines build variants
if (WIN32)
Expand Down
4 changes: 4 additions & 0 deletions Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
FROM quay.io/pypa/manylinux2014_x86_64:latest
RUN (yum install cmake swig freeglut-devel zlib-devel -y)
ADD . / io/
CMD (cd io && /opt/python/cp38-cp38/bin/python setup.py bdist_wheel && auditwheel repair dist/partio-1.0.0-cp38-cp38-linux_x86_64.whl)
1 change: 1 addition & 0 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -53,6 +53,7 @@ doc: $(builddir)/stamp

$(builddir)/stamp:
mkdir -p $(builddir)
$(info CMAKE_FLAGS is $(CMAKE_FLAGS))
cd $(builddir) && cmake $(CMAKE_FLAGS) ../..
touch $@

Expand Down
51 changes: 26 additions & 25 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,22 +1,19 @@
[Partio](https://wdas.github.io/partio) - A library for particle IO and manipulation
=============================================================================================================
# [Partio](https://wdas.github.io/partio) - A library for particle IO and manipulation

This is the initial source code release of partio a tool we used for particle
reading/writing. It started out as an abstraction for the commonalities in
reading/writing. It started out as an abstraction for the commonalities in
particle models (i.e. accessing many attributes associated with an index or
entity).

Super impatient building guide
==============================
# Super impatient building guide

# Install Location ~ adjust accordingly
prefix=$HOME/local
git clone https://github.com/wdas/partio.git
cd partio
make -j prefix=$prefix install

Getting Started
===============
# Getting Started

CMake is used to build the project, but we provide a top-level Makefile
for convenience that takes care of all the steps.
Expand All @@ -29,8 +26,7 @@ with a temporary staging directory of `/tmp/stage` is:

make DESTDIR=/tmp/stage prefix=/usr/local install

Source code overview
====================
# Source code overview

src/
lib/ Library code (public API in root)
Expand All @@ -44,8 +40,7 @@ Source code overview
partinfo <particle file>
partview <particle file>

Class Model
-----------
## Class Model

The goal of the library is to abstract the particle interface from the data
representation. That is why Partio represents particles using three classes that
Expand All @@ -71,14 +66,13 @@ The functions used to get particle access are these:
returns ParticlesDataMutable
allows read/write access

Behind the scenes you could implement these classes however you like. Headers
only representation is called core/ParticleHeader.{h,cpp}. Simple
Behind the scenes you could implement these classes however you like. Headers
only representation is called core/ParticleHeader.{h,cpp}. Simple
non-interleaved attributes is core/ParticleSimple.{h,cpp}.

Attribute Data Model
--------------------
## Attribute Data Model

All particles have the same data attributes. They have the model that they are
All particles have the same data attributes. They have the model that they are
of three basic types with a count of how many scalar values they have.

VECTOR[3]
Expand All @@ -90,9 +84,7 @@ of three basic types with a count of how many scalar values they have.

This seems to encompass the most common file formats for particles


Iterating
---------
## Iterating

There are multiple ways to access data in the API. Here are
some tips
Expand All @@ -107,23 +99,32 @@ some tips
- Use iterators to do linear operations over all particles They are much more
optimized than both data() and the dataAsFloat or


Backends
--------
## Backends

Behind the scenes there are SimpleParticles, ParticleHeaders, and
SimpleParticlesInterleaved. In the future I would like to write a disk-based
SimpleParticlesInterleaved. In the future I would like to write a disk-based
cached back end that can dynamically only load the data that is necessary.
create(), read() and readCached could be augmented to create different
structures in these cases.

Readers/Writers
---------------
## Readers/Writers

New readers and writers can be added in the io/ directory. You simply need to
implement the interface ParticlesInfo, ParticlesData and ParticlesDataMutable
(or as many as you need). Editing the io/readers.h to add prototypes and
io/ParticleIO.cpp to add file extension bindings should be easy.

## Building the python Package for PyPi

To the partio for python and publish it to we have to build it using docker and upload it to PyPi.

```bash
# build the docker
docker build -t partio .
# run the build
docker run partio
# use twine to upload to pypi
twine upload wheelhouse/*
```

- Andrew Selle, Walt Disney Animation Studios
3 changes: 3 additions & 0 deletions setup.cfg
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
# Inside of setup.cfg
[metadata]
description-file = README.md
174 changes: 134 additions & 40 deletions setup.py
Original file line number Diff line number Diff line change
@@ -1,62 +1,156 @@
#!/usr/bin/env python
# -*- coding:utf-8 -*-

from setuptools import setup
from cmake_build_extension import BuildExtension, CMakeExtension
from setuptools import setup, Extension
from setuptools.command.install_lib import install_lib
from setuptools.command.build_ext import build_ext

# from cmake_build_extension import BuildExtension, CMakeExtension
import shutil
import subprocess
import platform

from pathlib import Path
import os
import subprocess
import shutil
import sys
from distutils.sysconfig import get_python_inc, get_config_var


class CMakeExtension(Extension):
def __init__(
self,
name,
source_dir: str = str(Path(".").absolute()),
cmake_configure_options=[],
):
super().__init__(name=name, sources=[])
self.cmake_configure_options = cmake_configure_options
self.cmake_build_type = "Release"
self.source_dir = source_dir


class BuildCMakeExtension(build_ext):
def run(self):
cmake_extensions = [e for e in self.extensions if isinstance(e, CMakeExtension)]
if len(cmake_extensions) == 0:
raise ValueError("No CMakeExtension objects found")

# Check that CMake is installed
if shutil.which("cmake") is None:
raise RuntimeError("Required command 'cmake' not found")

for ext in cmake_extensions:
self.build_extension(ext)

def build_extension(self, ext: Extension):
# CMake configure arguments
configure_args = [
f"-DCMAKE_BUILD_TYPE={ext.cmake_build_type}",
# "-GNinja",
f"-DCMAKE_INSTALL_PREFIX:PATH=build/install",
]
build_args = ["--config", ext.cmake_build_type]
# Extend the configure arguments with those passed from the extension
configure_args += ext.cmake_configure_options

if platform.system() == "Windows":
configure_args += []
elif platform.system() in {"Linux", "Darwin"}:
configure_args += []
else:
raise RuntimeError(f"Unsupported '{platform.system()}' platform")

# Get the absolute path to the build folder
build_folder = str(Path(".").absolute() / f"{self.build_temp}_{ext.name}")

# Make sure that the build folder exists
Path(build_folder).mkdir(exist_ok=True, parents=True)

# 1. Compose CMake configure command
configure_command = [
"cmake",
"-S",
ext.source_dir,
"-B",
build_folder,
] + configure_args

# 2. Compose CMake build command
build_command = ["cmake", "--build", build_folder] + build_args

# 3. Compose CMake install command
install_command = ["cmake", "--install", build_folder]

# create the output dirs
os.makedirs("build_partio/src/py", exist_ok=True)
print("")
print("==> Configuring:")
print(f"$ {' '.join(configure_command)}")
print("")
print("==> Building:")
print(f"$ {' '.join(build_command)}")
print("")
print("==> Installing:")
print(f"$ {' '.join(install_command)}")
print("")

PARTIO_EXT = CMakeExtension(
# Call CMake
subprocess.check_call(configure_command)
subprocess.check_call(build_command)
subprocess.check_call(install_command)


PARTIO_EXT = CMakeExtension(
name="partio",
install_prefix="partio",
source_dir=".",
cmake_configure_options=[
f"-DPYTHON_INCLUDE_DIR={get_python_inc()}",
f"-DPYTHON_LIBRARY={get_config_var('LIBDIR')} ",
f"-DPYTHON_EXECUTABLE={sys.executable} ",
],
)

class BuildPartioExtension(BuildExtension):
def __init__(self, *args, **kwars) -> None:
super().__init__(*args, **kwars)

def run(self) -> None:
super().run()
if shutil.which("patchelf") is None:
raise RuntimeError("Required command 'patchelf' not found")
class InstallLibs(install_lib):
def run(self):
self.announce("+++ InstallLibs", level=3)
self.skip_build = True

def build_extension(self, ext: CMakeExtension) -> None:
self.build_temp = "build"
super().build_extension(ext)
# Get the absolute path to the build folder
build_folder = str(Path(".").absolute() / f"{self.build_temp}_{ext.name}")
shared_object_path = os.path.join(build_folder, "src", "py", "_partio.so")
# directly set the path of the shared object into the shared object created from swig
patch_command = [
"patchelf",
"--replace-needed",
"libpartio.so.1",
f"{build_folder}/src/lib/libpartio.so.1",
shared_object_path
# Everything under self.build_dir will get moved into site-packages
# so move all things we want installed there
lib_dir = self.build_dir
write_dir = Path(lib_dir) / "partio"

os.makedirs(write_dir, exist_ok=True)

# copy equired files
python_version = f"python{sys.version_info[0]}.{sys.version_info[1]}"
base_path = Path.cwd() / "build" / "install" / "lib64"
to_install = [
base_path / "libpartio.so.1",
base_path / python_version / "site-packages" / "partio.py",
base_path / python_version / "site-packages" / "_partio.so",
]
print("Patching shared obbject to directly include the libpartio file")
print(f" => {patch_command}")
subprocess.check_call(patch_command)
for lib in to_install:
filename = Path(lib).name
target = str(write_dir / filename)
if os.path.isfile(target):
os.remove(target)
shutil.move(str(lib), str(write_dir))

# create init file
f = open(str(write_dir / "__init__.py"), "w")
f.write("from .partio import *")
f.close()
self.announce("+++ custom done", level=3)
super().run()


setup(
name='partio',
description='A Python wrapper for partio',
name="partio",
description="A Python wrapper for partio",
version="1.0.0",
packages=[],
package_dir={'': 'build_partio/src/py'},
ext_modules=[PARTIO_EXT],
cmdclass={
'build_ext': BuildPartioExtension
},
install_requires=["cmake-build-extension"],
python_requires='>=3.4',
cmdclass={"build_ext": BuildCMakeExtension, "install_lib": InstallLibs,},
python_requires=">=3.4",
)

)

0 comments on commit 74ad1ff

Please sign in to comment.