Abstract
We propose a novel method for action detection based on a new action descriptor called a shape flow that represents both the shape and movement of an object in a holistic and parsimonious manner. We find actions by finding shape flows in a target video that are similar to a template shape flow. Shape flows are largely independent of appearance, and the match cost function that we propose is invariant to scale changes and smooth nonlinear deformation in space and time. The problem of matching shape flows is difficult, however, yielding a large, non-convex, integer program. We propose a novel relaxation method based on successive convexification that converts this hard program into a vastly smaller linear program: By using only those variables that appear on the 4D lower convex hull of the matching cost volume, most of the variables in the linear program may be eliminated. Experiments confirm that the proposed shape flow method can successfully detect complex actions in cluttered video, even with self-occlusion, camera motion, and intra-class variation.
Chapter PDF
Similar content being viewed by others
Keywords
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.
References
Besag, J.: On the Statistical Analysis of Dirty Pictures. Journal of the Royal Statistical Society, Series B 48(3), 259–302 (1986)
Blank, M., Gorelick, L., Shechtman, E., Irani, M., Basri, R.: Actions as Space-Time Shapes. In: ICCV (2005)
Bobick, A., Davis, J.: The Recognition of Human Movement Using Temporal Templates. IEEE Trans. PAMI 23(3), 257–267 (2001)
Efros, A., Berg, A., Mori, G., Malik, J.: Recognizing Action at a Distance. In: ICCV (2003)
Interrante, V., Grosch, C.: Visualizing 3D Flow. IEEE Computer Graphics and Applications 18(4), 49–53 (1998)
Jiang, H., Drew, M.S., Li, Z.N.: Matching by Linear Programming and Successive Convexification. IEEE Trans. on PAMI 29(6), 959–975 (2007)
Ke, Y., Sukthankar, R., Hebert, M.: Event Detection in Crowded Videos. In: ICCV (2007)
Kenwright, D., Mallinson, G.: A 3-D Streamline Tracking Algorithm Using Dual Stream Functions. Visualization, 62–68 (1992)
Laptev, I., Lindeberg, T.: Space-Time Interest Points. In: ICCV (2003)
Laptev, I., Prez, P.: Retrieving Actions in Movies. In: ICCV (2007)
Mori, G., Ren, X.F., Efros, A., Malik, J.: Recovering Human Body Configurations: Combining Segmentation and Recognition. CVPR, 326–333 (2004)
Parameswaran, V., Chellappa, R.: Human Action-Recognition Using Mutual Invariants. CVIU 98(2), 295–325 (2005)
Ramanan, D., Forsyth, D.A., Zisserman, A.: Tracking People by Learning Their Appearance. IEEE Trans. on PAMI 29(1), 65–81 (2007)
Schuldt, C., Laptev, I., Caputo, B.: Recognizing Human Actions: A Local SVM Approach. In: ICPR (2004)
Scovanner, P., Ali, S., Shah, M.: A 3-Dimensional SIFT Descriptor and its Application to Action Recognition. ACM Multimedia (September 2007)
Shechtman, E., Irani, M.: Space-Time Behavior Based Correlation. In: CVPR (2005)
Sheikh, Y., Shah, M.: Exploring the Space of an Action for Human Action Recognition. In: ICCV (2005)
Weinland, D., Ronfard, R., Boyer, E.: Free Viewpoint Action Recognition using Motion History Volumes. In: CVIU (November/December 2006)
Yilmaz, A., Shah, M.: Actions as Objects: A Novel Action Representation. In: CVPR (2005)
Yilmaz, A., Shah, M.: Recognizing Human Actions in Videos Acquired by Uncalibrated Moving Cameras. In: ICCV (2005)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2008 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Jiang, H., Martin, D.R. (2008). Finding Actions Using Shape Flows. In: Forsyth, D., Torr, P., Zisserman, A. (eds) Computer Vision – ECCV 2008. ECCV 2008. Lecture Notes in Computer Science, vol 5303. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-88688-4_21
Download citation
DOI: https://doi.org/10.1007/978-3-540-88688-4_21
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-88685-3
Online ISBN: 978-3-540-88688-4
eBook Packages: Computer ScienceComputer Science (R0)