Review: Pedestrian Behavior Analysis and Trajectory Prediction with Deep Learning

Main Article Content

M Bhavsingh
K Samunnisa


It is becoming increasingly necessary for artificially intelligent systems to be able to monitor, evaluate, and anticipate the actions of humans as more of these systems are deployed in human-populated places. For an autonomous vehicle to make intelligent navigation decisions, a comprehensive analysis of the movement patterns of surrounding traffic agents and precise projections of their future trajectories are required. In this paper, we investigate how to assess the behavior of pedestrians and predict their trajectories using a unified deep learning model. Specifically, we look at how to do both of these things. Investigate the methods that were utilized to collect data and evaluate performance, as well as any surprises, challenges, insights, and recommendations that occurred as a result of the investigation's findings

Article Details

How to Cite
M Bhavsingh, B.Pannalal, and K Samunnisa, “Review: Pedestrian Behavior Analysis and Trajectory Prediction with Deep Learning”, Int. J. Comput. Eng. Res. Trends, vol. 9, no. 12, pp. 263–268, Dec. 2022.


Korbmacher, R., & Tordeux, A. : Review of Pedestrian Trajectory Prediction Methods: Comparing Deep Learning and Knowledge-based Approaches. ArXiv, abs /(2022). 2111.06740.

Yi, S., Li, H., Wang, X.: Understanding pedestrian behaviors from stationary crowd groups. In: Proceedings of CVPR (2019)

Yi, S., Li, H., Wang, X.: Pedestrian behavior modeling from stationary crowds with applications to intelligent surveillance. TIP 25(9), 4354–4368 (2016).

Tang, S., Andriluka, M., Milan, A., Schindler, K., Roth, S., Schiele, B.: Learning people detectors for tracking in crowded scenes. In: Proceedings of ICCV (2016)

Leal-Taix´e, L., Fenzi, M., Kuznetsova, A., Rosenhahn, B., Savarese, S.: Learning an image-based motion context for multiple people tracking. In: Proceedings of CVPR (2017)

Zhou, B., Wang, X., Tang, X.: Understanding collective crowd behaviors: learning a mixture model of dynamic pedestrian-agents. In: Proceedings of CVPR (2019)

Chang, M.C., Krahnstoever, N., Ge, W.: Probabilistic group-level motion analysis and scenario recognition. In: Proceedings of ICCV (2018).

Mehran, R., Oyama, A., Shah, M.: Abnormal crowd behavior detection using social force model. In: Proceedings of CVPR (2017)

Mahadevan, V., Li, W., Bhalodia, V., Vasconcelos, N.: Anomaly detection in crowded scenes. In: Proceedings of CVPR (2010)

Yi, S., Wang, X., Lu, C., Jia, J., Li, H.: L0 regularized stationary-time estimation for crowd analysis. TPAMI PP(99), 1 (2016).

Pellegrini, S., Ess, A., Schindler, K., Van Gool, L.: You’ll never walk alone: modeling social behavior for multi-target tracking. In: Proceedings of ICCV (2019)

Xiaoge Zhang, Sankaran Mahadevan, Bayesian neural networks for flight trajectory prediction and safety assessment, Decision Support Systems, Volume 131,2020,113246, ISSN 0167-9236.

S. Lefèvre, C. Laugier and J. Ibañez-Guzmán, "Exploiting map information for driver intention estimation at road intersections," 2011 IEEE Intelligent Vehicles Symposium (IV), Baden-Baden, 2011, pp. 583-588, doi: 10.1109/IVS.2011.5940452.

S. Danielsson, L. Petersson and A. Eidehall, "Monte Carlo based Threat Assessment: Analysis and Improvements," 2007 IEEE Intelligent Vehicles Symposium, Istanbul, 2007, pp. 233-238, doi: 10.1109/IVS.2007.4290120.

Q. Tran and J. Firl, "A probabilistic discriminative approach for situation recognition in traffic scenarios," 2012 IEEE Intelligent Vehicles Symposium, Alcala de Henares, 2012, pp. 147-152, doi: 10.1109/IVS.2012.6232279.

C. E. Rasmussen & C. K. I. Williams, Gaussian Processes for Machine Learning, the MIT Press, 2006, ISBN 026218253X.

Wenjie Luo, Bin Yang, and Raquel Urtasun. Fast and furious: Real time end-to-end 3d detection, tracking and motion forecasting with a single convolutional net. In Proceedings of the IEEE conference on Computer Vision and Pattern Recognition, pages 3569–3577, 2018.

Amir Sadeghian, Vineet Kosaraju, Ali Sadeghian, Noriaki Hirose, Hamid Rezatofighi, and Silvio Savarese. Sophie: An attentive gan for predicting paths compliant to social and physical constraints. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 1349– 1358, 2019.

Nachiket Deo and Mohan M Trivedi. Convolutional social pooling for vehicle trajectory prediction. arXiv preprint arXiv:1805.06771, 2018.

Rohan Chandra, Uttaran Bhattacharya, Aniket Bera, and Dinesh Manocha. Traphic: Trajectory prediction in dense and heterogeneous traffic using weighted interactions. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 8483–8492, 2019

Rohan Chandra, Uttaran Bhattacharya, Christian Roncal, Aniket Bera, and Dinesh Manocha. Robusttp: End-toend trajectory prediction for heterogeneous road-agents in dense traffic with noisy sensor inputs. arXiv preprint arXiv:1907.08752, 2019.

Yuexin Ma, Xinge Zhu, Sibo Zhang, Ruigang Yang, Wenping Wang, and Dinesh Manocha. Trafficpredict: Trajectory prediction for heterogeneous traffic-agents. arXiv preprint arXiv:1811.02146, 2018.

N. Djuric, V. Radosavljevic, H. Cui, T. Nguyen, F.-C. Chou, T.-H. Lin, and J. Schneider. Short-term Motion Prediction of Traffic Actors for Autonomous Driving using Deep Convolutional Networks. ArXiv e-prints, Aug. 2018.

Shengnan Guo, Youfang Lin, Ning Feng, Chao Song, and Huaiyu Wan. Attention based spatial-temporal graph convolutional networks for traffic flow forecasting. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 33, pages 922–929, 2019.

Zhiyong Cui, Kristian Henrickson, Ruimin Ke, and Yinhai Wang. Traffic graph convolutional recurrent neural network: A deep learning framework for network-scale traffic learning and forecasting. arXiv preprint arXiv:1802.07007, 2018.

Kooij, Julian & Schneider, Nicolas & Flohr, Fabian & Gavrila, Dariu. (2014). Context-Based Pedestrian Path Prediction. 10.1007/978-3-319-10599-4_40.

S. K. Jayaraman, C. Creech, L. P. Robert Jr., D. M. Tilbury, X. J. Yang, A. K. Pradhan, and K. M. Tsui, “Trust in AV: An Uncertainty Reduction Model of AV-Pedestrian Interactions,” in Companion 2018 ACM/IEEE Int. Conf. Human-Robot Interaction, 2018, pp. 133–134.

L. Robert, “The future of pedestrian–automated vehicle interactions,” XRDS: Crossroads, ACM, vol. 25, no. 3, 2019.

K. Saleh, M. Hossny, and S. Nahavandi, “Towards trusted autonomous vehicles from vulnerable road users perspective,” in 2017 11th Annu. IEEE Int. Syst. Conf., SysCon, 2017, pp. 1–7.

A. Gorrini, G. Vizzari, and S. Bandini, “Towards Modelling Pedestrian-Vehicle Interactions: Empirical Study on Urban Unsignalized Intersection,” arXiv preprint arXiv:1610.07892, 2016.

A. Alahi, K. Goel, V. Ramanathan, A. Robicquet, L. Fei-Fei, and S. Savarese, “Social LSTM: Human Trajectory Prediction in Crowded Spaces,” in Proc. IEEE Conf. Comput. Vision Pattern Recognition, 2016, pp. 961–971.

Yi, S., Li, H., & Wang, X. (2016). Pedestrian Behavior Understanding and Prediction with Deep Neural Networks. Lecture Notes in Computer Science, 263–279.

Basharat, A., Gritai, A., Shah, M.: Learning object motion patterns for anomaly detection and improved object detection. In: Proceedings of CVPR (2008)

Long, J., Shelhamer, E., Darrell, T.: Fully convolutional networks for semantic segmentation. In: Proceedings of CVPR (2015)

Simonyan, K., Zisserman, A.: Two-stream convolutional networks for action recognition in videos. In: Proceedings of NIPS (2014)

Shao, J., Kang, K., Loy, C.C., Wang, X.: Deeply learned attributes for crowded scene understanding. In: Proceedings of CVPR (2015)