Abstract
Landmark-based human action recognition in videos is a challenging task in computer vision. One key step is to design a generic approach that generates discriminative features for the spatial structure and temporal dynamics. To this end, we regard the evolving landmark data as a high-dimensional path and apply path signature techniques to provide an expressive, robust, non-linear, and interpretable representation for the sequential events. We do not extract signature features from the raw path, rather we propose path disintegrations and path transformations as preprocessing steps. Path disintegrations turn a high-dimensional path linearly into a collection of lower-dimensional paths; some of these paths are in pose space while others are defined over a multi-scale collection of temporal intervals. Path transformations decorate the paths with additional coordinates in standard ways to allow the truncated signatures of transformed paths to expose additional features. For spatial representation, we apply the non-linear signature transform to vectorize the paths that arise out of pose disintegration, and for temporal representation, we apply it again to describe this evolving vectorization. Finally, all the features are joined together to constitute the input vector of a linear single-hidden-layer fully-connected network for classification. Experimental results on four diverse datasets demonstrated that the proposed feature set with only a linear shallow network is effective and achieves comparable state-of-the-art results to the advanced deep networks, and meanwhile, is capable of interpretation.
Access provided by Autonomous University of Puebla. Download to read the full chapter text
Chapter PDF
Similar content being viewed by others
Change history
21 June 2022
The original version of the book was inadvertently published with incorrect abstracts in the chapters. This has now been amended.
In addition to this, the affiliation of author Dr. Bertram Tschiderer has been changed to Faculty of Mathematics, University of Vienna in the online version of Chapter 10.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this chapter
Cite this chapter
Yang, W., Lyons, T., Ni, H., Schmid, C., Jin, L. (2022). Developing the Path Signature Methodology and Its Application to Landmark- Based Human Action Recognition. In: Yin, G., Zariphopoulou, T. (eds) Stochastic Analysis, Filtering, and Stochastic Optimization. Springer, Cham. https://doi.org/10.1007/978-3-030-98519-6_18
Download citation
DOI: https://doi.org/10.1007/978-3-030-98519-6_18
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-98518-9
Online ISBN: 978-3-030-98519-6
eBook Packages: Mathematics and StatisticsMathematics and Statistics (R0)