OPEN ACCESS
Dense motion field estimation is a key computer vision problem. Many solutions have been proposed to compute small or large displacements, narrow or wide baseline stereo disparity, or non-rigid surface registration, but a unified methodology is still lacking. We here introduce a general framework that robustly combines direct and feature-based matching. The feature-based cost is built around a novel robust distance function that handles keypoints and weak features such as segments. It allows us to use putative feature matches to guide dense motion estimation out of local minima. Our framework uses a robust direct data term with a powerful second order regularization. Our framework achieves state of the art performance in several cases (standard optical flow benchmarks, wide-baseline stereo and non-rigid surface registration).
RÉSUMÉ
L’estimation dense de correspondances entre deux images est un sujet essentiel de la vision par ordinateur et s’exprime sous plusieurs formes : déformations rigides ou flexibles avec de faibles ou grandes amplitudes de déplacements. De nombreuses solutions spécifiques existent mais aucune méthodologie unifiée n’a été formulée. Cet article propose une nouvelle approche générale qui combine de manière robuste un coût dense par pixel et un coût basé sur des correspondances de primitives. Ce dernier utilise une distance robuste permettant d’exploiter des correspondances de points ou de segments. Les correspondances permettent d’empêcher l’optimisation dense de tomber dans un minimum local. En utilisant un coût dense robuste, associé à une régularisation au second ordre et une détection explicite des auto-occultations, nous obtenons des résultats égalant ou surpassant l’état de l’art pour les applications de flot optique 2D, stéréo à fortes disparité et recalage de surfaces déformables.
optical flow, stereo, matching, features
MOTS-CLÉS
flot optique, stéréo, correspondances, primitives
Alcantarilla P. F., Bartoli A. (2012). Deformable 3d reconstruction with an object database. In British machine vision conference (bmvc), p. 1–12.
Baker S., Scharstein D., Lewis J., Roth S., Black M., Szeliski R. (2011). A database and evaluation methodology for optical flow. International Journal of Computer Vision (IJCV).
Bay H., Tuytelaars T., Van Gool L. (2006). SURF: Speeded up robust features. In European conference on computer vision (eccv).
Bradski G. (2000). The OpenCV library. Dr. Dobb’s Journal of Software Tools.
Bredies K. (2012). Recovering piecewise smooth multichannel images by minimization of convex functionals with total generalized variation penalty. SFB Report, vol. 6.
Bredies K., Kunisch K., Pock T. (2010). Total generalized variation. SIAM Journal on Imaging Sciences.
Brox T., Malik J. (2011). Large displacement optical flow: descriptor matching in variational motion estimation. International Journal of Pattern Analysis and Machine Intelligence (PAMI).
Chambolle A., Pock T. (2011). A first-order primal-dual algorithm for convex problems with applications to imaging. Journal of Mathematical Imaging and Vision.
Gay-Bellile V., Bartoli A., Sayd P. (2010). Direct estimation of nonrigid registrations with imagebased self-occlusion reasoning. International Journal of Pattern Analysis and Machine Intelligence (PAMI).
Geiger A., Lenz P., Urtasun R. (2012). Are we ready for autonomous driving? The KITTI vision benchmark suite. In Computer vision and pattern recognition (cvpr).
Hartley R., Zisserman A. (2000). Multiple view geometry in computer vision (vol. 2). Cambridge Univ Press.
Horn B. K. P., Schunck B. G. (1981). Determining optical flow. Artificial Intelligence.
Lowe D. G. (2004, novembre). Distinctive image features from scale-invariant keypoints. Int. J. Comput. Vision, vol. 60, no 2, p. 91–110.
Mei X., Sun X., Zhou M., Jiao S., Wang H., Zhang X. (2011). On building an accurate stereo matching system on graphics hardware. In Third iccv workshop on gpus for computer vision.
Perriollat M., Bartoli A. (2012). A computational model of bounded developable surfaces with application to image-based three-dimensional reconstruction. Computer Animation and Virtual Worlds.
Pilet J., Lepetit V., Fua P. (2008). Fast non-rigid surface detection, registration and realistic augmentation. International Journal of Computer Vision (IJCV).
Pizarro D., Bartoli A. (2012). Feature-based deformable surface detection with self-occlusion reasoning. International Journal of Computer Vision (IJCV).
Ranftl R., Gehrig S., Pock T., Bischof H. (2012). Pushing the limits of stereo using variational stereo estimation. In Intelligent vehicles symposium (iv).
Rosten E., Porter R., Drummond T. (2010). Faster and better: A machine learning approach to corner detection. International Journal of Pattern Analysis and Machine Intelligence (PAMI).
Special session on robust optical flow. (2013). (German Conference on Pattern Recognition)
Tola E., Lepetit V., Fua P. (2010). Daisy: An efficient dense descriptor applied to wide-baseline stereo. Pattern Analysis and Machine Intelligence, IEEE Transactions on, vol. 32, no 5.
Wang L., Neumann U., You S. (2009). Wide-baseline image matching using line signatures. In International conference on computer vision (iccv).
Werlberger M., Trobin W., Pock T., Wedel A., Cremers D., Bischof H. (2009). Anisotropic Huber-L1 optical flow. In British machine vision conference (bmvc).
Xu L., Jia J., Matsushita Y. (2010). Motion detail preserving optical flow estimation. In Computer vision and pattern recognition (cvpr).
Zabih R., Woodfill J. (1994). Non-parametric local transforms for computing visual correspondence. In European conference on computer vision (eccv).