PyOP2 on cx1

This page gives a detailed tutorial on how to build PyOP2 on the Imperial College CX1 compute cluster and run the provided example applications.

Building PyOP2 on cx1

Build OP2-Common

Follow the detailed instructions in the OP2-Common wiki.

Get PyOP2

git clone git://github.com/OP2/PyOP2.git
cd PyOP2

Set up your environment

Create a file .env with the following content (adjust OP2_DIR to your OP2-Common installation location):

module load cmake/2.8.7
module load cuda/4.0.17
module load intel-suite/11.1
module load git/1.7.10
module load mpi/intel-3.1
module load python/2.7.3
module load opencl/intel-2.0

export OP2_DIR=/path/to/OP2-Common/op2

and source it with . .env.

Set up your virtualenv and pip

The easiest way to install PyOP2 dependencies is by setting up a virtualenv, which also make sure these packages don't interfere with your regular Python environment.

Install virtualenv (if you don't have it already):

git clone git://github.com/pypa/virtualenv.git
cd virtualenv/
python setup.py install --user

Create a virtualenv in your PyOP2 tree:

python -m virtualenv --distribute .venv

and activate it:

. .venv/bin/activate

You can now use pip to install packages in your virtualenv!

To leave the virutalenv, use deactivate.

Install dependencies

pip install cython decorator pyyaml pytest codepy ply jinja2 mako numpy
pip install https://launchpad.net/instant/1.0.x/1.0.0/+download/instant-1.0.0.tar.gz
pip install hg+https://bitbucket.org/fr710/pycparser#egg=pycparser-2.08

PETSC

LD_PRELOAD= CC= CXX= FC= F77= F90= \
  PETSC_CONFIGURE_OPTIONS="--with-fortran-interfaces=1 --with-c++-support --with-openmp --download-f-blas-lapack=1 CC=mpicc CXX=mpicxx FC=mpifc F77=mpif77 F90=mpif90" \
  pip install hg+https://bitbucket.org/fr710/petsc-3.3-omp
pip install hg+https://bitbucket.org/fr710/petsc4py#egg=petsc4py

Note: Make sure the bzr/2.5.0 module is not loaded when you're building PETSc. Otherwise the PETSc build system will pick up the bzrlib commands module instead of the commands module from the Python standard library.

PyCUDA

git clone http://git.tiker.net/trees/pycuda.git
cd pycuda
git submodule init
git submodule update
./configure
python setup.py build
python setup.py install

PyOpenCL

Since the OpenCL headers and library are in a non-standard location, PyOpenCL needs to be configured and installed by hand:

git clone http://git.tiker.net/trees/pyopencl.git
cd pyopencl
./configure.py --cl-inc-dir=${OPENCL_HOME}/usr/include --cl-lib-dir=${OPENCL_HOME}/usr/lib64
python setup.py install