The widespread adoption of Large Language Models (LLMs) has brought increasing attention to their vulnerability to jailbreak attacks, where adversarial prompts are crafted to bypass safety mechanisms and induce harmful responses. While proprietary models such as GPT-4 have undergone extensive safety evaluations, the robustness of recently released open-source models, including the DeepSeek family, remains less well understood despite their rapid integration into real-world LLM applications. This paper presents a systematic evaluation of jailbreak robustness for the DeepSeek-R1 distilled model family, providing a standardized comparison with GPT-3.5 and GPT-4 using the HarmBench benchmark (v1.0). Our evaluation covers seven representative jailbreak strategies across 320 harmful behaviors from the HarmBench text test split, spanning both functional categories and semantic dimensions. The results show that DeepSeek exhibits certain resistance to automated attacks, particularly LLM-guided approaches such as TAP-T (Tree of Attacks with Pruning–Transfer), but remains more vulnerable to prompt-level and manually constructed adversarial attacks. In comparison, GPT-4 Turbo demonstrates stronger and more consistent safety alignment across diverse harmful behaviors, suggesting the benefit of more extensive safety optimization and reinforcement learning from human feedback. Beyond aggregate attack success rates, our behavioral analysis and case studies reveal that DeepSeek does not consistently enforce safety constraints under adversarial conditions, resulting in irregular refusal patterns across different attack settings. These findings expose a broader challenge in balancing model capability with alignment robustness and motivate the development of more targeted safety tuning techniques for reliable deployment of open-source LLMs.
- Open Access
- Article
SoK: A Comprehensive Security Analysis of Jailbreak Resilience in GPT and DeepSeek Models †
- Xiaodong Wu 1,‡,
- Xiangman Li 1,‡,
- Qi Li 1,
- Lingshuang Liu 2,
- Jianbing Ni 1,*
Author Information
Received: 22 Jul 2026 | Revised: 06 Aug 2026 | Accepted: 11 Aug 2026 | Published: 21 Aug 2026
Abstract
Keywords
References
- 1.
Radford, A.; Narasimhan, K.; Salimans, T.; et al. Improving Language Understanding by Generative Pre-Training. 2018. Available online: https://www.mikecaptain.com/resources/pdf/GPT-1.pdf (accessed on 11 August 2026).
- 2.
Achiam, J.; Adler, S.; Agarwal, S.; et al. GPT-4 technical report. arXiv 2023. arXiv:2303.08774.
- 3.
Bi, X.; Chen, D.; Chen, G.; et al. DeepSeek LLM: Scaling open-source language models with longtermism. arXiv 2024. arXiv:2401.02954.
- 4.
Guo, D.; Yang, D.; Zhang, H.; et al. DeepSeek-R1: Incentivizing reasoning capability in LLMs via reinforcement learning. arXiv 2025. arXiv:2501.12948.
- 5.
Sitawarin, C.; Mu, N.; Wagner, D.; et al. PAL: Proxy-guided black-box attack on large language models. arXiv 2024. arXiv:2402.09674.
- 6.
Zou, A.; Wang, Z.; Carlini, N.; et al. Universal and transferable adversarial attacks on aligned language models. arXiv 2023. arXiv:2307.15043.
- 7.
Wei, A.; Haghtalab, N.; Steinhardt, J. Jailbroken: How does LLM safety training fail? arXiv 2023. arXiv:2311.06607.
- 8.
Zhao, S.; Jia, M.; Guo, Z.; et al. A survey of backdoor attacks and defenses on large language models: Implications for security measures. TechRxiv 2024. https://doi.org/10.36227/techrxiv.172832726.62863760/v1.
- 9.
Ouyang, L.; Wu, J.; Jiang, X.; et al. Training language models to follow instructions with human feedback. In Proceedings of the Advances in Neural Information Processing Systems 35 (NeurIPS 2022), New Orleans, LA, USA, 28 November–9 December 2022; pp. 27730–27744.
- 10.
Cao, Y.; Gu, N.; Shen, X.; et al. Defending large language models against jailbreak attacks through chain of thought prompting. In Proceedings of the 2024 International Conference on Networking and Network Applications (NaNA), Yinchuan, China, 9–12 August 2024; pp. 125–130.
- 11.
Anil, C.; Durmus, E.; Panickssery, N.; et al. Many-shot jailbreaking. In Proceedings of the Advances in Neural Information Processing Systems 37 (NeurIPS 2024), Vancouver, BC, Canada, 10–15 December 2024; pp. 129696–129742.
- 12.
Deng, G.; Liu, Y.; Wang, K.; et al. Pandora: Jailbreak GPTs by retrieval augmented generation poisoning. arXiv 2024. arXiv:2402.08416.
- 13.
Li, H.; Guo, D.; Fan, W.; et al. Multi-step jailbreaking privacy attacks on ChatGPT. arXiv 2023. arXiv:2304.05197.
- 14.
Wang, J.; Liu, Z.; Park, K.H.; et al. Adversarial demonstration attacks on large language models. arXiv 2023. arXiv:2305.14950.
- 15.
Wei, Z.; Wang, Y.; Li, A.; et al. Jailbreak and guard aligned language models with only few in-context demonstrations. arXiv 2023. arXiv:2310.06387.
- 16.
Zheng, X.; Pang, T.; Du, C.; et al. Improved few-shot jailbreaking can circumvent aligned language models and their defenses. In Proceedings of the Advances in Neural Information Processing Systems 37 (NeurIPS 2024), Vancouver, BC, Canada, 10–15 December 2024; pp. 32856–32887.
- 17.
Chang, Z.; Li, M.; Liu, Y.; et al. Play guessing game with LLM: Indirect jailbreak attack with implicit clues. arXiv 2024. arXiv:2402.09091.
- 18.
Jiang, F.; Xu, Z.; Niu, L.; et al. ArtPrompt: ASCII art-based jailbreak attacks against aligned LLMs. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), Bangkok, Thailand, 11–16 August 2024; pp. 15157–15173.
- 19.
Li, X.; Liang, S.; Zhang, J.; et al. Semantic mirror jailbreak: Genetic algorithm based jailbreak prompts against open-source LLMs. arXiv 2024. arXiv:2402.14872.
- 20.
Lin, R.; Han, B.; Li, F.; et al. Understanding and enhancing the transferability of jailbreaking attacks. arXiv 2025. arXiv:2502.03052.
- 21.
Liu, X.; Xu, N.; Chen, M.; et al. AutoDAN: Generating stealthy jailbreak prompts on aligned large language models. arXiv 2023. arXiv:2310.04451.
- 22.
Liu, Y.; He, X.; Xiong, M.; et al. FlipAttack: Jailbreak LLMs via flipping. In Proceedings of the Forty-Second International Conference on Machine Learning, Vancouver, BC, Canada, 13–19 July 2025; pp. 38623–38663.
- 23.
Takemoto, K. All in how you ask for it: Simple black-box method for jailbreak attacks. Appl. Sci. 2024, 14, 3558.
- 24.
Wei, Z.; Liu, Y.; Erichson, N.B. Emoji attack: Enhancing jailbreak attacks against judge LLM detection. In Proceedings of the Forty-Second International Conference on Machine Learning, Vancouver, BC, Canada, 13–19 July 2025; pp. 66103–66117.
- 25.
Yang, X.; Tang, X.; Han, J.; et al. The dark side of trust: Authority citation-driven jailbreak attacks on large language models. arXiv 2024. arXiv:2411.11407.
- 26.
Yu, J.; Lin, X.; Yu, Z.; et al. GPTFUZZER: Red teaming large language models with auto-generated jailbreak prompts. arXiv 2023. arXiv:2309.10253.
- 27.
Yuan, Y.; Jiao, W.; Wang, W.; et al. GPT-4 is too smart to be safe: Stealthy chat with LLMs via cipher. In Proceedings of the International Conference on Learning Representations 2024 (ICLR 2024), Vienna, Austria, 7–11 May 2024.
- 28.
Casper, S.; Lin, J.; Kwon, J.; et al. Explore, establish, exploit: Red teaming language models from scratch. arXiv 2023. arXiv:2306.09442.
- 29.
Chao, P.; Robey, A.; Dobriban, E.; et al. Jailbreaking black box large language models in twenty queries. In Proceedings of the 2025 IEEE Conference on Secure and Trustworthy Machine Learning (SaTML), Copenhagen, Denmark, 9–11 April 2025; pp. 23–42.
- 30.
Deng, G.; Liu, Y.; Li, Y.; et al. MasterKey: Automated jailbreak across multiple large language model chatbots. arXiv 2023. arXiv:2307.08715.
- 31.
Ge, S.; Zhou, C.; Hou, R.; et al. MART: Improving LLM safety with multi-round automatic red-teaming. In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers), Mexico City, Mexico, 16–21 June 2024; pp. 1927–1937.
- 32.
Jin, H.; Chen, R.; Zhang, P.; et al. GUARD: Role-playing to generate natural-language jailbreakings to test guideline adherence of large language models. arXiv 2024. arXiv:2402.03299.
- 33.
Liu, C.; Zhao, F.; Qing, L.; et al. Goal-oriented prompt attack and safety evaluation for LLMs. arXiv 2023. arXiv:2309.11830.
- 34.
Mehrotra, A.; Zampetakis, M.; Kassianik, P.; et al. Tree of attacks: Jailbreaking black-box LLMs automatically. In Proceedings of the Advances in Neural Information Processing Systems 37 (NeurIPS 2024), Vancouver, BC, Canada, 10–15 December 2024; pp. 61065–61105.
- 35.
Shah, R.; Feuillade-Montixi, Q.; Pour, S.; et al. Scalable and transferable black-box jailbreaks for language models via persona modulation. arXiv 2023. arXiv:2311.03348.
- 36.
Tian, Y.; Yang, X.; Zhang, J.; et al. Evil geniuses: Delving into the safety of LLM-based agents. arXiv 2023. arXiv:2311.11855.
- 37.
Zeng, Y.; Lin, H.; Zhang, J.; et al. How Johnny can persuade LLMs to jailbreak them: Rethinking persuasion to challenge AI safety by humanizing LLMs. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), Bangkok, Thailand, 11–16 August 2024; pp. 14322–14350.
- 38.
Geisler, S.; Wollschlager, T.; Abdalla, M.H.I.; et al. Attacking large language models with projected gradient descent. arXiv 2024. arXiv:2402.09154.
- 39.
Hayase, J.; Borevkovic, E.; Carlini, N.; et al. Query-based adversarial prompt generation. In Proceedings of the Advances in Neural Information Processing Systems 37 (NeurIPS 2024), Vancouver, BC, Canada, 10–15 December 2024; pp. 128260–128279.
- 40.
Jia, X.; Pang, T.; Du, C.; et al. Improved techniques for optimization-based jailbreaking on large language models. arXiv 2024. arXiv:2405.21018.
- 41.
Sun, C.E.; Liu, X.; Yang, W.; et al. Iterative self-tuning LLMs for enhanced jailbreaking capabilities. arXiv 2024. arXiv:2410.18469.
- 42.
Wang, H.; Li, H.; Huang, M.; et al. From noise to clarity: Unraveling the adversarial suffix of large language model attacks via translation of text embeddings. arXiv 2024. arXiv:2402.16006.
- 43.
Yang, J.; Zhang, Z.; Cui, S.; et al. Guiding not forcing: Enhancing the transferability of jailbreaking attacks on LLMs via removing superfluous constraints. arXiv 2025. arXiv:2503.01865.
- 44.
Zhu, S.; Zhang, R.; An, B.; et al. AutoDAN: Interpretable gradient-based adversarial attacks on large language models. arXiv 2023. arXiv:2310.15140.
- 45.
Du, Y.; Zhao, S.; Ma, M.; et al. Analyzing the inherent response tendency of LLMs: Real-world instructions-driven jailbreak. arXiv 2023. arXiv:2312.04127.
- 46.
Guo, X.; Yu, F.; Zhang, H.; et al. COLD-attack: Jailbreaking LLMs with stealthiness and controllability. In Proceedings of the 41st International Conference on Machine Learning, Vienna, Austria, 21–27 July 2024; Vol. 235, pp. 16974–17002.
- 47.
Hu, L.; Wang, B. DROJ: A prompt-driven attack against large language models. arXiv 2024. arXiv:2411.09125.
- 48.
Huang, Y.; Gupta, S.; Xia, M.; et al. Catastrophic jailbreak of open-source LLMs via exploiting generation. In Proceedings of the Twelfth International Conference on Learning Representations, Vienna, Austria, 7–11 May 2024.
- 49.
Zhang, Z.; Shen, G.; Tao, G.; et al. Make them spill the beans! coercive knowledge extraction from (production) LLMs. arXiv 2023. arXiv:2312.04782.
- 50.
Zhao, X.; Yang, X.; Pang, T.; et al. Weak-to-strong jailbreaking on large language models. arXiv 2024. arXiv:2401.17256.
- 51.
Zhou, Y.; Lou, J.; Huang, Z.; et al. Don’t say no: Jailbreaking LLM by suppressing refusal. arXiv 2024. arXiv:2404.16369.
- 52.
Qi, X.; Zeng, Y.; Xie, T.; et al. Fine-tuning aligned language models compromises safety, even when users do not intend to! arXiv 2023. arXiv:2310.03693.
- 53.
Yang, X.; Wang, X.; Zhang, Q.; et al. Shadow alignment: The ease of subverting safely-aligned language models. arXiv 2023. arXiv:2310.02949.
- 54.
Zhan, Q.; Fang, R.; Bindu, R.; et al. Removing RLHF protections in GPT-4 via fine-tuning. arXiv 2023. arXiv:2311.05553.
- 55.
Mazeika, M.; Phan, L.; Yin, X.; et al. HarmBench: A standardized evaluation framework for automated red teaming and robust refusal. arXiv 2024. arXiv:2402.04249.
- 56.
Zhou, W.; Wang, X.; Xiong, L.; et al. EasyJailbreak: A unified framework for jailbreaking large language models. arXiv 2024. arXiv:2403.12171.
- 57.
Peng, Y.; Long, Z.; Dong, F.; et al. Playing language game with LLMs leads to jailbreaking. arXiv 2024. arXiv:2411.12762.
- 58.
Shen, X.; Chen, Z.; Backes, M.; et al. “Do anything now”: Characterizing and evaluating in-the-wild jailbreak prompts on large language models. In Proceedings of the CCS’24: ACM SIGSAC Conference on Computer and Communications Security, Salt Lake City, UT, USA, 14–18 October 2024; pp. 1671–1685.
- 59.
Chang, W.; Zhu, T.; Zhao, Y.; et al. Chain-of-Lure: A synthetic narrative-driven approach to compromise large language models. arXiv 2025. arXiv:2505.17519.
- 60.
Ying, Z.; Zhang, D.; Jing, Z.; et al. Reasoning-augmented conversation for multi-turn jailbreak attacks on large language models. arXiv 2025. arXiv:2502.11054.
- 61.
Qi, S.; Zou, Y.; Li, P.; et al. Amplified vulnerabilities: Structured jailbreak attacks on LLM-based multi-agent debate. arXiv 2025. arXiv:2504.16489.
- 62.
Kuo, M.; Zhang, J.; Ding, A.; et al. H-CoT: Hijacking the chain-of-thought safety reasoning mechanism to jailbreak large reasoning models, including OpenAI O1/O3, DeepSeek-R1, and Gemini 2.0 Flash thinking. arXiv 2025. arXiv:2502.12893.
- 63.
Ahmed, M.; Abdelmouty, M.; Kim, M.; et al. Advancing jailbreak strategies: A hybrid approach to exploiting LLM vulnerabilities and bypassing modern defenses. arXiv 2025. arXiv:2506.21972.
- 64.
Hu, X.; Chen, P.Y.; Ho, T.Y. Gradient cuff: Detecting jailbreak attacks on large language models by exploring refusal loss landscapes. In Proceedings of the Advances in Neural Information Processing Systems 37 (NeurIPS 2024), Vancouver, BC, Canada, 10–15 December 2024; pp. 126265–126296.
- 65.
Zhang, S.; Zhai, Y.; Guo, K.; et al. JBShield: Defending large language models from jailbreak attacks through activated concept analysis and manipulation. In Proceedings of the 34th USENIX Security Symposium (USENIX Security 25), Seattle, WA, USA, 13–15 August 2025; pp. 8215–8234.
- 66.
Zhou, K.; Liu, C.; Zhao, X.; et al. The hidden risks of large reasoning models: A safety assessment of R1. arXiv 2025. arXiv:2502.12659.
This work is licensed under a Creative Commons Attribution 4.0 International License.
Suite 4002 Level 4, 447 Collins Street, Melbourne, Victoria 3000, Australia
General Inquiries: info@sciltp.com