JAIA/
Articles/
2610005407
  • Open Access
  • Article

Robust Deep Long-Term Correlation Filter Tracking

  • Hao Dou 1,   
  • Xiwang Guo 1 and 
  • Qiangqiang Wu 2,*

Received: 15 May 2026 | Revised: 19 September 2026 | Accepted: 13 October 2026 | Published: 23 October 2026

Abstract

Long-term object tracking is an essential task for developing robust vision ability for intelligent robots and vehicles. To cope with various challenges in the dynamic driving environment, existing methods usually maintain a target appearance model and update it in an online manner. However, the errors may accumulate during the updating process when the target suffers from the problems caused by such issues as long-term occlusion and out-of-view, thus resulting in tracking failures. This work proposes a Robust Deep Long-term Tracking (RDLT) method, which can alleviate the impact of erroneous updates by incorporating an offline learned deep Siamese network into a unified correlation filter tracking framework. Specifically, it learns a deep correlation filter model that models target temporal variations to locate the target. To avoid tracking failures caused by the challenges like long-term occlusion, a pre-trained deep Siamese network is activated to search for the target on an enlarged searching region when the correlation filter outputs low response values, by which several target candidate windows can be obtained. To correctly select the optimal target window, this work further proposes a novel multi-expert evaluation schema. It evaluates the target candidate windows by using a pair of “experts”, i.e., Correlation filter-based expert (C-expert) and Deep similarity-based expert (D-expert). Both experts jointly evaluate all the target candidate windows and select the optimal one as the final tracking result, thereby avoiding the correlation filter being greatly affected by corrupt samples. Extensive experiments on three popular benchmarks including OTB-2013, OTB-2015 and UAV20L, show that the proposed RDLT method performs favorably against the other state-of-the-art methods.

References 

  • 1.

    Rakha, H.A.; Ahn, K.; Faris, W.; et al. Simple Vehicle Powertrain Model for Modeling Intelligent Vehicle Applications. IEEE Trans. Intell. Transp. Syst. 2012, 13, 770–780. https://doi.org/10.1109/tits.2012.2188517.

  • 2.

    Balid, W.; Tafish, H.; Refai, H.H. Intelligent Vehicle Counting and Classification Sensor for Real-Time Traffic Surveillance. IEEE Trans. Intell. Transp. Syst. 2018, 19, 1784–1794. https://doi.org/10.1109/tits.2017.2741507.

  • 3.

    Cao, Z.; Li, J.; Shao, S.; et al. Siamese Adaptive Network-Based Accurate and Robust Visual Object Tracking Algorithm for Quadrupedal Robots. IEEE Trans. Cybern. 2025, 55, 1264–1276. https://doi.org/10.1109/tcyb.2025.3527342.

  • 4.

    Cao, Z.; Li, J.; Zhang, D.; et al. A Multi-Object Tracking Algorithm With Center-Based Feature Extraction and Occlusion Handling. IEEE Trans. Intell. Transp. Syst. 2023, 24, 4464–4473. https://doi.org/10.1109/tits.2022.3229978.

  • 5.

    Wu, Y.; Lim, J.; Yang, M.H. Object Tracking Benchmark. IEEE Trans. Pattern Anal. Mach. Intell. 2015, 37, 1834–1848. https://doi.org/10.1109/tpami.2014.2388226.

  • 6.

    Ross, D.A.; Lim, J.; Lin, R.S.; et al. Incremental Learning for Robust Visual Tracking. Int. J. Comput. Vis. 2008, 77, 125–141. https://doi.org/10.1007/s11263-007-0075-7.

  • 7.

    Henriques, J.F.; Caseiro, R.; Martins, P.; et al. High-Speed Tracking with Kernelized Correlation Filters. IEEE Trans. Pattern Anal. Mach. Intell. 2015, 37, 583–596. https://doi.org/10.1109/tpami.2014.2345390.

  • 8.

    Kalal, Z.; Mikolajczyk, K.; Matas, J. Tracking-Learning-Detection. IEEE Trans. Pattern Anal. Mach. Intell. 2012, 34, 1409–1422. https://doi.org/10.1109/tpami.2011.239.

  • 9.

    Hong, Z.; Chen, Z.; Wang, C.; et al. MUlti-Store Tracker (MUSTer): A Cognitive Psychology Inspired Approach to Object Tracking. In Proceedings of the 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Boston, MA, USA, 7–12 June 2015; pp. 749–758. https://doi.org/10.1109/cvpr.2015.7298675.

  • 10.

    Ma, C.; Huang, J.B.; Yang, X.; et al. Hierarchical Convolutional Features for Visual Tracking. In Proceedings of the 2015 IEEE International Conference on Computer Vision (ICCV), Santiago, Chile, 7–13 December 2015; pp. 3074–3082. https://doi.org/10.1109/iccv.2015.352.

  • 11.

    Qi, Y.; Zhang, S.; Qin, L.; et al. Hedged Deep Tracking. In Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA, 27–30 June 2016; pp. 4303–4311. https://doi.org/10.1109/cvpr.2016.466.

  • 12.

    Danelljan, M.; Hager, G.; Khan, F.S.; et al. Convolutional Features for Correlation Filter Based Visual Tracking. In Proceedings of the 2015 IEEE International Conference on Computer Vision Workshop (ICCVW), Santiago, Chile, 7–13 December 2015; pp. 621–629. https://doi.org/10.1109/iccvw.2015.84.

  • 13.

    Wu, Y.; Lim, J.; Yang, M.H. Online Object Tracking: A Benchmark. In Proceedings of the 2013 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Portland, OR, USA, 23–28 June 2013; pp. 2411–2418. https://doi.org/10.1109/cvpr.2013.312.

  • 14.

    Ma, C.; Yang, X.; Zhang, C.; et al. Long-Term Correlation Tracking. In Proceedings of the 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Boston, MA, USA, 7–12 June 2015; pp. 5388–5396. https://doi.org/10.1109/cvpr.2015.7299177.

  • 15.

    Tao, R.; Gavves, E.; Smeulders, A.W.M. Siamese Instance Search for Tracking. In Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA, 27–30 June 2016; pp. 1420–1429. https://doi.org/10.1109/cvpr.2016.158.

  • 16.

    Zhang, L.; Han, H.; Zhou, M.; et al. An Improved Discriminative Model Prediction Approach to Real-Time Tracking of Objects With Camera as Sensors. IEEE Sens. J. 2021, 21, 17308–17317. https://doi.org/10.1109/jsen.2021.3079214.

  • 17.

    Bertinetto, L.; Valmadre, J.; Henriques, J.F.; et al. Fully-Convolutional Siamese Networks for Object Tracking. In Computer Vision, Proceedings of the ECCV 2016 Workshops, Amsterdam, The Netherlands, 8–10 and 15–16 October 2016; Springer International Publishing: Cham, Switzerland, 2016; pp. 850–865. https://doi.org/10.1007/978-3-319-48881-3_56.

  • 18.

    Mueller, M.; Smith, N.; Ghanem, B. A Benchmark and Simulator for UAV Tracking. In Computer Vision, Proceedings of the ECCV 2016, Amsterdam, The Netherlands, 11–14 October 2016; Springer International Publishing: Cham, Switzerland, 2016; pp. 445–461. https://doi.org/10.1007/978-3-319-46448-0_27.

  • 19.

    Bolme, D.; Beveridge, J.R.; Draper, B.A.; et al. Visual Object Tracking Using Adaptive Correlation Filters. In Proceedings of the 2010 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), San Francisco, CA, USA, 13–18 June 2010; pp. 2544–2550. https://doi.org/10.1109/cvpr.2010.5539960.

  • 20.

    Henriques, J.F.; Caseiro, R.; Martins, P.; et al. Exploiting the Circulant Structure of Tracking-by-Detection with Kernels. In Computer Vision, Proceedings of the ECCV 2012, Florence, Italy, 7–12 October 2012; Springer: Berlin/Heidelberg, Germany, 2012; pp. 702–715. https://doi.org/10.1007/978-3-642-33765-9_50.

  • 21.

    Simonyan, K.; Zisserman, A. Very Deep Convolutional Networks for Large-Scale Image Recognition. arXiv 2014, arXiv:1409.1556v6.

  • 22.

    Zhang, K.; Zhang, L.; Yang, M.H. Real-Time Compressive Tracking. In Computer Vision, Proceedings of the ECCV 2012, Florence, Italy, 7–12 October 2012; Springer: Berlin/Heidelberg, Germany, 2012; pp. 864–877. https://doi.org/10.1007/978-3-642-33712-3_62.

  • 23.

    Chaudhuri, K.; Freund, Y.; Hsu, D. A Parameter-Free Hedging Algorithm. arXiv 2009, arXiv:0903.2851v2.

  • 24.

    Cui, Y.; Guo, D.; Shao, Y.; et al. Joint Classification and Regression for Visual Tracking with Fully Convolutional Siamese Networks. Int. J. Comput. Vis. 2022, 130, 550–566. https://doi.org/10.1007/s11263-021-01559-4.

  • 25.

    Chen, Z.; Zhong, B.; Li, G.; et al. Siamese Box Adaptive Network for Visual Tracking. In Proceedings of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, WA, USA, 13–19 June 2020; pp. 6667–6676. https://doi.org/10.1109/cvpr42600.2020.00670.

  • 26.

    Vaswani, A.; Shazeer, N.; Parmar, N.; et al. Attention Is All You Need. In Proceedings of the Advances in Neural Information Processing Systems (NeurIPS), Long Beach, CA, USA, 4–9 December 2017; pp. 5998–6008.

  • 27.

    Dosovitskiy, A.; Beyer, L.; Kolesnikov, A.; et al. An Image is Worth 16×16 Words: Transformers for Image Recognition at Scale. In Proceedings of the International Conference on Learning Representations (ICLR), Virtual, 3–7 May 2021.

  • 28.

    Chen, X.; Yan, B.; Zhu, J.; et al. Transformer Tracking. In Proceedings of the 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Nashville, TN, USA, 20–25 June 2021; pp. 8122–8131. https://doi.org/10.1109/cvpr46437.2021.00803.

  • 29.

    Yan, B.; Peng, H.; Fu, J.; et al. Learning Spatio-Temporal Transformer for Visual Tracking. In Proceedings of the 2021 IEEE/CVF International Conference on Computer Vision (ICCV), Montreal, QC, Canada, 10–17 October 2021; pp. 10428–10437. https://doi.org/10.1109/iccv48922.2021.01028.

  • 30.

    Wang, N.; Zhou, W.; Wang, J.; et al. Transformer Meets Tracker: Exploiting Temporal Context for Robust Visual Tracking. In Proceedings of the 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Nashville, TN, USA, 20–25 June 2021; pp. 1571–1580. https://doi.org/10.1109/cvpr46437.2021.00162.

  • 31.

    Cui, Y.; Jiang, C.; Wang, L.; et al. MixFormer: End-to-End Tracking with Iterative Mixed Attention. In Proceedings of the 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), New Orleans, LA, USA, 18–24 June 2022; pp. 13598–13608. https://doi.org/10.1109/cvpr52688.2022.01324.

  • 32.

    Ye, B.; Chang, H.; Ma, B.; et al. Joint Feature Learning and Relation Modeling for Tracking: A One-Stream Framework. In Proceedings of the European Conference on Computer Vision (ECCV), Tel Aviv, Israel, 23–27 October 2022; pp. 341–357.

  • 33.

    Mayer, C.; Danelljan, M.; Bhat, G.; et al. Transforming Model Prediction for Tracking. In Proceedings of the 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), New Orleans, LA, USA, 18–24 June 2022; pp. 8721–8730. https://doi.org/10.1109/cvpr52688.2022.00853.

  • 34.

    Lin, L.; Fan, H.; Zhang, Z.; et al. SwinTrack: A Simple and Strong Baseline for Transformer Tracking. In Proceedings of the Advances in Neural Information Processing Systems, New Orleans, LA, USA, 28 November–9 December 2022; Volume 35, pp. 16743–16754.

  • 35.

    Chen, X.; Peng, H.; Wang, D.; et al. SeqTrack: Sequence to Sequence Learning for Visual Object Tracking. In Proceedings of the 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Vancouver, BC, Canada, 17–24 June 2023; pp. 14572–14581. https://doi.org/10.1109/cvpr52729.2023.01400.

  • 36.

    Wu, Q.; Yang, T.; Liu, Z.; et al. DropMAE: Masked Autoencoders with Spatial-Attention Dropout for Tracking Tasks. In Proceedings of the 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Vancouver, BC, Canada, 17–24 June 2023; pp. 14561–14571. https://doi.org/10.1109/cvpr52729.2023.01399.

  • 37.

    Wu, Q.; Yang, T.; Fang, B.; et al. M2P: Improving Visual Foundation Models with Mask-to-Point Weakly-Supervised Learning for Dense Point Tracking. arXiv 2026, arXiv:2603.17813v1.

  • 38.

    Wu, Q.; Chan, A.B. Learning Tracking Representations from Single Point Annotations. In Proceedings of the 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), Seattle, WA, USA, 17–18 June 2024; pp. 2606–2615. https://doi.org/10.1109/cvprw63382.2024.00267.

  • 39.

    Wu, Q.; Wan, J.; Chan, A.B. Progressive Unsupervised Learning for Visual Object Tracking. In Proceedings of the 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Nashville, TN, USA, 20–25 June 2021; pp. 2992–3001. https://doi.org/10.1109/cvpr46437.2021.00301.

  • 40.

    Wu, Q.; Yang, T.; Wu, W.; et al. Scalable Video Object Segmentation with Simplified Framework. In Proceedings of the 2023 IEEE/CVF International Conference on Computer Vision (ICCV), Paris, France, 1–6 October 2023; pp. 13833–13843. https://doi.org/10.1109/iccv51070.2023.01276.

  • 41.

    Wu, Q.; Yan, Y.; Liang, Y.; et al. DSNet: Deep and Shallow Feature Learning for Efficient Visual Tracking. In Computer Vision, Proceedings of the ACCV 2018, Perth, WA, Australia, 2–6 December 2018; Springer International Publishing: Cham, Switzerland, 2019; pp. 119–134. https://doi.org/10.1007/978-3-030-20873-8_8.

  • 42.

    Wu, Q.; Chan, A.B. Meta-Graph Adaptation for Visual Object Tracking. In Proceedings of the 2021 IEEE International Conference on Multimedia and Expo (ICME), Shenzhen, China, 5–9 July 2021; pp. 1–6. https://doi.org/10.1109/icme51207.2021.9428441.

  • 43.

    Wei, X.; Bai, Y.; Zheng, Y.; et al. Autoregressive Visual Tracking. In Proceedings of the 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Vancouver, BC, Canada, 17–24 June 2023; pp. 9697–9706. https://doi.org/10.1109/cvpr52729.2023.00935.

  • 44.

    Bai, Y.; Zhao, Z.; Gong, Y.; et al. ARTrackV2: Prompting Autoregressive Tracker Where to Look and How to Describe. In Proceedings of the 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, WA, USA, 16–22 June 2024; pp. 19048–19057. https://doi.org/10.1109/cvpr52733.2024.01802.

  • 45.

    Zheng, Y.; Zhong, B.; Liang, Q.; et al. ODTrack: Online Dense Temporal Token Learning for Visual Tracking. In Proceedings of the AAAI Conference on Artificial Intelligence, Vancouver, BC, Canada, 20–27 February 2024; Volume 38, pp. 7588–7596.

  • 46.

    Xie, J.; Zhong, B.; Mo, Z.; et al. Autoregressive Queries for Adaptive Tracking with Spatio-Temporal Transformers. In Proceedings of the 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, WA, USA, 16–22 June 2024; pp. 19300–19309. https://doi.org/10.1109/cvpr52733.2024.01826.

  • 47.

    Kristan, M.; Matas, J.; Danelljan, M.; et al. The First Visual Object Tracking Segmentation VOTS2023 Challenge Results. In Proceedings of the 2023 IEEE/CVF International Conference on Computer Vision Workshops (ICCVW), Paris, France, 2–6 October 2023; pp. 1788–1810. https://doi.org/10.1109/iccvw60793.2023.00195.

  • 48.

    Kristan, M.; Matas, J.; Tokmakov, P.; et al. The Second Visual Object Tracking Segmentation VOTS2024 Challenge Results. In Computer Vision, Proceedings of the ECCV 2024 Workshops, Milan, Italy, 29 September–4 October 2024; Springer Nature: Cham, Switzerland, 2025; pp. 357–383. https://doi.org/10.1007/978-3-031-91767-7_24.

  • 49.

    Wu, Q.; Yu, Y.; Kong, C.; et al. Temporal Unlearnable Examples: Preventing Personal Video Data from Unauthorized Exploitation by Object Tracking. In Proceedings of the 2025 IEEE/CVF International Conference on Computer Vision (ICCV), Honolulu, HI, USA, 19–25 October 2025; pp. 11110–11121. https://doi.org/10.1109/iccv51701.2025.01034.

  • 50.

    Danelljan, M.; Hager, G.; Khan, F.S.; et al. Discriminative Scale Space Tracking. IEEE Trans. Pattern Anal. Mach. Intell. 2017, 39, 1561–1575. https://doi.org/10.1109/tpami.2016.2609928.

  • 51.

    Danelljan, M.; Hager, G.; Khan, F.S.; et al. Learning Spatially Regularized Correlation Filters for Visual Tracking. In Proceedings of the 2015 IEEE International Conference on Computer Vision (ICCV), Santiago, Chile, 7–13 December 2015; pp. 4310–4318. https://doi.org/10.1109/iccv.2015.490.

  • 52.

    Bertinetto, L.; Valmadre, J.; Golodetz, S.; et al. Staple: Complementary Learners for Real-Time Tracking. In Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA, 27–30 June 2016; pp. 1401–1409. https://doi.org/10.1109/cvpr.2016.156.

  • 53.

    Lukezic, A.; Vojır, T.; Cehovin Zajc, L.; et al. Discriminative Correlation Filter with Channel and Spatial Reliability. In Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA, 21–26 July 2017; pp. 4847–4856. https://doi.org/10.1109/cvpr.2017.515.

  • 54.

    Galoogahi, H.K.; Fagg, A.; Lucey, S. Learning Background-Aware Correlation Filters for Visual Tracking. In Proceedings of the 2017 IEEE International Conference on Computer Vision (ICCV), Venice, Italy, 22–29 October 2017; pp. 1144–1152. https://doi.org/10.1109/iccv.2017.129.

  • 55.

    Zhang, T.; Xu, C.; Yang, M.H. Multi-Task Correlation Particle Filter for Robust Object Tracking. In Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA, 21–26 July 2017; pp. 4819–4827. https://doi.org/10.1109/cvpr.2017.512.

  • 56.

    Deng, J.; Dong, W.; Socher, R.; et al. ImageNet: A Large-Scale Hierarchical Image Database. In Proceedings of the 2009 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Miami, FL, USA, 20–25 June 2009; pp. 248–255. https://doi.org/10.1109/cvpr.2009.5206848.

  • 57.

    Huang, D.; Luo, L.; Chen, Z.; et al. Applying Detection Proposals to Visual Tracking for Scale and Aspect Ratio Adaptability. Int. J. Comput. Vis. 2017, 122, 524–541. https://doi.org/10.1007/s11263-016-0974-6.

  • 58.

    Zhang, J.; Ma, S.; Sclaroff, S. MEEM: Robust Tracking via Multiple Experts Using Entropy Minimization. In Proceedings of the European Conference on Computer Vision (ECCV), Zurich, Switzerland, 6–12 September 2014; pp. 188–203. https://doi.org/10.1007/978-3-319-10599-4_13.

Share this article:
Download PDF
DOI
10.53941/jaia.2026.100018
Issue
Volume 1, Issue 3
How to Cite
Dou, H.; Guo, X.; Wu, Q. Robust Deep Long-Term Correlation Filter Tracking. Journal of Artificial Intelligence for Automation 2026, 1 (3), 18. https://doi.org/10.53941/jaia.2026.100018.
RIS
BibTex
Copyright & License
article copyright Image
Copyright (c) 2026 by the authors.
Article Metrics
19
Article Views
0
Citations