Optical Flow Algorithm Resources

A curated list of resources dedicated to optical flow algorithms. Any suggestions and pull requests are welcome.

Papers & Code

Classical methods

• [1981-IJCAI, Lucas-Kanade method] An iterative image registration technique with an application to stereo vision paper
• [1981-AI, Horn-Schunck method] Determining optical flow paper
• [2003-SCIA, Farneback flow] Two-frame Motion Estimation Based on Polynomial Expansion paper code
• [2004-ECCV, Brox method] High Accuracy Optical Flow Estimation Based on a Theory for Warping paper code
• [2005-IJCV] Lucas/Kanade meets Horn/Schunck: Combining local and global optic flow methods paper
• [2007-DAGM, TVL1 method] A duality based approach for realtime tv-l1 optical flow paper code
• [2011-TPAMI, LDOF flow] Large displacement optical flow: descriptor matching in variational motion estimation paper code
• [2013-ICCV, deep flow] DeepFlow: Large Displacement Optical Flow with Deep Matching paper homepage code
• [2016-ECCV, DIS flow] Fast Optical Flow using Dense Inverse Search paper code

Deep learning based methods

• [2015-ICCV, FlowNet1] FlowNet: Learning Optical Flow with Convolutional Networks paper new code old code
• [2017-CVPR, FlowNet2] FlowNet 2.0: Evolution of Optical Flow Estimation with Deep Networks paper code homepage
• [2020-ECCV, best paper, RAFT] RAFT: Recurrent All Pairs Field Transforms for Optical Flow paper code

Optical flow in severe environment (haze, rain)

• [2017-Xiv] Robust Optical Flow Estimation in Rainy Scenes paper

Others

• [2010-ECCV] Dense point trajectories by GPU-accelerated large displacement optical flow paper code

Optical flow toolkit

• [Li Routeng's toolbox] Python-based optical flow toolkit for existing popular dataset code
• [EzFlow] A modular PyTorch library for optical flow estimation using neural networks code documentation

Datasets

• Middlebury 2009 paper
  • 8 image pairs for training, with ground truth flows generated using four different techniques
  • Displacements are very small, typi- cally below 10 pixels.
• KITTI 2012 paper
  • 194 training image pairs, large displacements, contains only a very special motion type
  • The ground truth is obtained from real world scenes by simultaneously recording the scenes with a camera and a 3D laser scanner.
  • Task: stereo, flow, sceneflow, depth, odometry, object, road, tracking, semantics, etc.
• MPI Sintel 2012 paper
  • 1041 training image pairs, ground truth from rendered artificial scenes with special attention to realistic image properties
  • Very long sequences, large motions, specular reflections, motion blur, defocus blur, atmospheric effects
  • Task: optical flow.
• Flying Chairs (Vision group, Uni-Freiburg) 2015 paper
  • 22872 image pairs, a synthetic dataset with optical flow ground truth
  • Task: optical flow.
• ChairsSDHom (Vision group, Uni-Freiburg) 2017 paper
  • Task: optical flow
  • Designed to be robust to untextured regions and to produce flow magnitude histograms close to those of the UCF101 dataset (small displacement, less than 1 pixel).

Open source implementation

• Optical Flow with OpenCV 3
• OpenCV 3 Optical Flow Algorithms in Extra Modules