A complete analysis of a geometric diagram hinges on interpreting both its fundamental primitives and the accompanying natural language text, yet existing models struggle to process the rich semantics within these descriptions, often leading to ambiguity and restricted reasoning. To address this, our work introduces a method that deeply integrates a Transformer-based text encoder within a sophisticated visual parsing architecture. Central to our approach is a novel Semantic-Guided Cross-Attention mechanism, which uses a global sentence representation as a semantic query to dynamically guide the model’s focus toward the most relevant visual primitives based on the textual context. This end-to-end process generates context-aware visual features that are then processed by a Graph Neural Network (GNN) to perform robust cross-modal reasoning. Validated on the large-scale PGDP5K and IMP-Geometry3K datasets, our method demonstrates substantial accuracy improvements in relationship parsing and geometric proposition generation, especially in challenging cases involving text-diagram ambiguity, and significantly surpasses current state-of-the-art baselines by offering a more effective framework for fusing deep textual semantics with visual information.
- Open Access
- Article
Enhancing Geometric Diagram Parsing with Deep Textual Semantics
- Xuhui Zhang 1,
- Yanli Wang 2,
- Pengpeng Jian 1,*,
- Lei Wu 1
Author Information
Received: 07 Nov 2025 | Revised: 18 Mar 2026 | Accepted: 25 Apr 2026 | Published: 30 Jun 2026
Abstract
Keywords
geometric problem solving | multimodal learning | geometric diagram parsing | cross-attention | DistilBERT
References
- 1.
Amini, A., Gabriel, S., Lin, S., Koncel-Kedziorski, R., Choi, Y., & Hajishirzi, H. (2019, June 2–7). MathQA: Towards interpretable math word problem solving with operation-based formalisms [Conference session]. 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) (pp. 2957–2967), Minneapolis, MN, USA.
- 2.
Antol, S., Agrawal, A., Lu, J., Mitchell, M., Batra, D., Zitnick, C. L., & Parikh, D. (2015, December 7–13). VQA: Visual question answering [Conference session]. 2015 IEEE International Conference on Computer Vision (ICCV) (pp. 2425–2433), Santiago, Chile. https://doi.org/10.1109/ICCV.2015.279.
- 3.
Chen, J., Tang, J., Qin, J., Xia, T., Leng, Z., Lin, X., Hao, Y., Wei, S., Ji, P., & Liang, X. (2021, August 1–6). GeoQA: A geometric question answering benchmark towards multimodal numerical reasoning [Conference session]. Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021 (pp. 4887–4898), Online.
- 4.
Chen, Y.-C., Li, L., Yu, L., El Kholy, A., Ahmed, F., Gan, Z., Cheng, Y., & Liu, J. (2020, August 23–28). UNITER: Universal image-text representation learning [Conference session]. 16th European Conference on Computer Vision (ECCV) (pp. 104–120), Glasgow, UK.
- 5.
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019, June 2–7). BERT: Pre-training of deep bidirectional transformers for language understanding [Conference session]. 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) (pp. 4171–4186), Minneapolis, MN, USA. https://doi.org/10.18653/v1/N19-1423.
- 6.
He, P., Liu, X., Gao, J., & Chen, W. (2020). DeBERTa: Decoding-enhanced BERT with disentangled attention. arXiv. https://doi.org/10.48550/arXiv.2006.03654
- 7.
Lin, T.-Y., Dollár, P., Girshick, R., & He, K. (2017, July 21–26). Feature pyramid networks for object detection [Conference session]. 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (pp. 2117–2125), Honolulu, HI, USA.
- 8.
Liu, Y., Ott, M., Goyal, N., Du, J., Joshi, M., Chen, D., Levy, O., Lewis, M., Zettlemoyer, L., & Stoyanov, V. (2019). RoBERTa: A robustly optimized BERT pretraining approach. arXiv. https://doi.org/10.48550/arXiv.1907.11692
- 9.
Lu, J., Batra, D., Parikh, D., & Lee, S. (2019, December 8–14). ViLBERT: Pretraining for ground-and-vision language tasks [Conference session]. Advances in Neural Information Processing Systems (NeurIPS 2019) (Vol. 32, pp. 1–12), Vancouver, BC, Canada.
- 10.
Lu, P., Qiu, L., Chen, J., Xia, T., Zhao, Y., Zhang, W., Zhou, Y., & Wu, Y. N. (2021, August 1–6). Inter-gps: Interpretable geometry problem solving with formal language and symbolic reasoning [Conference session]. 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (ACL-IJCNLP) (pp. 5946–5958), Virtual.
- 11.
Sanh, V., Debut, L., Chaumond, J., & Wolf, T. (2019). DistilBERT, a distilled version of BERT: Smaller, faster, cheaper and lighter. arXiv. https://doi.org/10.48550/arXiv.1910.01108.
- 12.
Seo, M. J., Hajishirzi, H., Farhadi, A., Etzioni, O., & Malcolm, C. (2015, September 17–21). Solving geometry problems: Combining text and diagram interpretation [Conference session]. 2015 Conference on Empirical Methods in Natural Language Processing (EMNLP) (pp. 1456–1466), Lisbon, Portugal.
- 13.
Su, W., Lin, X., Wang, W., & Li, J. (2019). VL-BERT: Pre-training of generic visual-linguistic representations. arXiv. https://doi.org/10.48550/arXiv.1908.08530.
- 14.
Tan, H., & Bansal, M. (2019, November 3–7). LXMERT: Learning cross-modality encoder representations from transformers [Conference session]. 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP) (pp. 5100–5111), Hong Kong, China.
- 15.
Tian, Z., Shen, C., Chen, H., & He, T. (2019, October 27–November 2). FCOS: Fully convolutional one-stage object detection [Conference session]. 2019 IEEE/CVF International Conference on Computer Vision (ICCV) (pp. 9627–9636), Seoul, Republic of Korea.
- 16.
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, Ł., & Polosukhin, I. (2017, December 4–9). Attention is all you need [Conference session]. Advances in Neural Information Processing Systems (NIPS 2017) (Vol. 30, pp. 5998–6008), Long Beach, CA, USA.
- 17.
Veličković, P., Cucurull, G., Casanova, A., Romero, A., Liò, P., & Bengio, Y. (2018, April 30–May 3). Graph attention networks [Conference session]. 2018 Conference on Learning Representations (ICLR), Vancouver, BC, Canada.
- 18.
Xu, D., Zhu, Y., Choy, C. B., & Li, F.-F. (2017, July 21–26). Scene graph generation by iterative message passing [Conference session]. 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (pp. 5410–5419), Honolulu, HI, USA.
- 19.
Zhang, M. L., Yin, F., Hao, Y. H., & Liu, C. L. (2022, July 23–29). Plane geometry diagram parsing [Conference session]. 31st International Joint Conference on Artificial Intelligence (IJCAI) (pp. 1636–1643), Vienna, Austria.
- 20.
Zhao, C. (2025). Rethinking authorship in the age of AI: Reflections on the AI-integrated writing framework (AWAI). Journal of Educational Technology and Innovation, 7(2), 25–38. https://doi.org/10.61414/h2a5bt21.
This work is licensed under a Creative Commons Attribution 4.0 International License.
Suite 4002 Level 4, 447 Collins Street, Melbourne, Victoria 3000, Australia
General Inquiries: info@sciltp.com