Adversarial Attacks on URL-Based Phishing Detection Models
Abstract
Phishing attacks remain one of the most widespread and economically significant cyberthreats. Machine and deep learning models for URL phishing detection achieve F1 scores of 0.95–0.99 on standard datasets, but their resilience to targeted adversarial modifications of input data remains poorly understood. This paper proposes a systematization of phishing attack types and detection methods, a comparative analysis of URL-based architectures (Random Forest, symbolic CNN, LSTM, CNN+LSTM), a classification of adversarial attacks on phishing detectors, and a Python framework that implements seven types of black-box evasion attacks: homoglyph substitution, typosquatting, subdomain injection, URL extension, URL encoding, GAN mutation with beam search, and composite chains. The developed framework satisfies the following requirements: generation of adversarial URLs for seven attack types; CLI with parameter groups for each type; Python API for embedding into automated pipelines; deterministic generation via a seed parameter; support for predefined and custom chains; visual highlighting of changes in verbose mode.
Experimental evaluation was performed on a dataset of 73,575 URLs (PhishTank + Marchal2014). Homoglyph modifications were shown to reduce the F1-score by 10–15% for all architectures; typosquatting by 9–14%; subdomain injection by 8% for Random Forest and 8–13% for DL models; URL extension by 10% for Random Forest and up to 4% for DL models; URL encoding by 6% for Random Forest and 0–10% for DL models (CNN+LSTM does not degrade). The full_evasion chain reduces F1 to 0.79–0.84 for all architectures; the maximum chain reduces it to 0.77–0.84. GAN mutation with heuristic scoring reduces F1 by 14–16% without access to model parameters. All attacks are transferable in black-box mode. Practical recommendations for improving the robustness of detectors are formulated: NFKC Unicode normalization, IDN/Punycode detector, adversarial training, Random Forest and DL ensembling, and URL length limitation. The framework's throughput is up to 220,000 URLs/min for single attacks and 48,000 URLs/min for three-stage chains.
Full Text:
PDF (Russian)References
Kaspersky Lab, “Kaspersky security bulletin 2024: Spam and spam-and-phishing-in-2024, 2024, accessed: 2025-01-10.
Anti-Phishing Working Group, “Phishing activity trends report q1 2024,” https://apwg.org/trendsreports/, 2024, accessed: 2025-01-15.
Federal Bureau of Investigation, “Internet crime report 2023,”https://www.ic3.gov/Media/PDF/AnnualReport/2023_IC3Report.pdf, 2024, accessed: 2025-01-20.
IBM Security, “Cost of a data breach report 2024,” https://www.ibm.com/reports/data-breach, 2024, accessed: 2025-01-20.
Verizon, “2024 data breach investigations report,” https://www.verizon.com/business/resources/reports/dbir/, 2024, accessed: 2025-01-15.
T. Holz, C. Gorecki, K. Rieck, and F. C. Freiling, “Measuring and detecting fast-flux service networks,” in Network and Distributed System Security Symposium (NDSS), 2008.
O. K. Sahingoz, E. Buber, O. Demir, and B. Diri, “Machine learning based phishing detection from URLs,” Expert Systems with Applications, vol. 117, pp. 345–357, 2019.
S. Y. Yerima and M. K. Alzaylaee, “PhishGuard: A convolutional neural network based model for detecting phishing URLs with explainability analysis,” arXiv preprint arXiv:2404.17960, 2024.
A. Anand, S. Garg, G. Choudhary, and N. Pandey, “AntiPhishStack: LSTM-based stacked generalization model for optimized phishing URL detection,” Symmetry, vol. 16, no. 2, p. 248, 2024.
G.Apruzzese,P.Laskov,andA.Tastemirova,“SpacePhish:The evasion-space of adversarial attacks against phishing website detectors using machine learning,” in Annual Computer Security Applications Conference (ACSAC), 2022, pp. 578–592.
B. Liang, J. Su, W. Guo, Y. Shi, and X. Zhang, “Directed adversarial sampling attacks on phishing detection,” Journal of Computer Security, vol. 29, no. 1, pp. 1–29, 2021.
R. Liu, Y. Lin, X. Yang, J. Y. Ng, D. M. Divakaran, and J. S. Dong, “PhishIntel: Toward practical deployment of reference-based phishing detection,” in ACM SIGSAC Conference on Computer and Communications Security (CCS), 2024.
G. Apruzzese, M. Colajanni, L. Ferretti, and M. Marchetti, “Revisiting the performance of machine learning-based phishing URL detection,” IEEE Transactions on Network and Service Management, vol. 21, no. 2, pp. 1631–1648, 2024.
A.SafiandS.Singh,“Asystematicliteraturereviewonphishing website detection techniques,” Journal of King Saud University – Computer and Information Sciences, vol. 35, no. 2, pp. 590– 611, 2023.
G. Apruzzese, P. Laskov, and A. Tastemirova, “ Multi-SpacePhish: Extending the evasion-space of adversarial attacks against phishing website detectors using machine learning,” ACM Transactions on Privacy and Security, 2023.
N. S. Afanasyeva, D. A. Elizarov, and T. A. Myznikova, “Klassifikatsiya fishingovykh atak i mery protivodeystviya im,” Informatsionnaya bezopasnost’ regionov, no. 5(89), 2022.
P. Kintis, N. Miramirkhani, C. Lever, Y. Chen, R. Romero-Gomez, N. Pitropakis, N. Provos, and M. Antonakakis, “Hiding in plain sight: A longitudinal study of combosquatting abuse,” in ACM SIGSAC Conference on Computer and Communications Security (CCS), 2017, pp. 569–586.
T. Holgers, D. E. Watson, and S. D. Gribble, “Cutting through the confusion: A measurement study of homograph attacks,” in USENIX Annual Technical Conference (ATC), 2006, pp. 261–266.
M. A. Adebowale, K. T. Lwin, E. Sanchez, and M. A. Hossain, “Intelligent phishing website detection system using modified frequent pattern-growth,” International Journal of Engineering and Advanced Technology, vol. 10, no. 3, pp. 22–29, 2021.
L. Tang and Q. H. Mahmoud, “A survey of machine learning-based solutions for phishing website detection,” Machine Learning and Knowledge Extraction, vol. 3, no. 3, pp. 672–694, 2021.
G. Apruzzese, M. Colajanni, L. Ferretti, and M. Marchetti, “Evasion attacks against banking fraud detection systems,” https://arxiv.org/abs/2004.06954, 2020, arXiv:2004.06954.
J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova, “BERT: Pre-training of deep bidirectional transformers for language understanding,” arXiv preprint arXiv:1810.04805, 2018.
T. Koide, D. Chiba, and M. Akiyama, “Detecting phishing sites using ChatGPT,” arXiv preprint arXiv:2306.05816, 2024.
S. Bell and P. Komisarczuk, “An analysis of phishing blacklists: Google Safe Browsing, OpenPhish, and PhishTank,” in Australasian Computer Science Week Multiconference (ACSW), 2020.
D.Sahoo,C.Liu,andS.C.Hoi,“MaliciousURLdetectionusing machine learning: A survey,” arXiv preprint arXiv:1701.07179, 2017.
Y. Zhang, J. Hong, and L. Cranor, “CANTINA: A content-based approach to detecting phishing web sites,” in Proceedings of the 16th International Conference on World Wide Web (WWW), 2007, pp. 639–648.
S. Abdelnabi, K. Krombholz, and M. Fritz, “VisualPhishNet: Zero-day phishing website detection by visual similarity,” in Proceedings of the 2020 ACM SIGSAC Conference on Computer and Communications Security (CCS), 2020, pp. 1681–1698.
Y. Lin, R. Liu, D. M. Divakaran, J. Y. Ng, Q. Z. Chan, Y. Lu, Y. Li, T. Wang, and J. S. Dong, “Phishpedia: A hybrid deep learning based approach to visually identify phishing webpages,” in USENIX Security Symposium, 2021, pp. 3793–3810.
T. Berners-Lee, R. Fielding, and L. Masinter, “RFC 3986: Uniform resource identifier (URI): Generic syntax,” IETF. https://www.rfc-editor.org/rfc/rfc3986, 2005.
F. Pedregosa, G. Varoquaux, A. Gramfort, V. Michel, B. Thirion, O. Grisel, M. Blondel, P. Prettenhofer, R. Weiss, V. Dubourg, J. Vanderplas, A. Passos, D. Cournapeau, M. Brucher, M. Perrot, and E. Duchesnay, “Scikit-learn: Machine learning in Python,” Journal of Machine Learning Research, vol. 12, pp. 2825–2830, 2011.
R. S. Rao and A. R. Pais, “Detection of phishing websites using an efficient feature-based machine learning framework,” Neural Computing and Applications, vol. 31, no. 8, pp. 3851–3873, 2019.
T. Chen and C. Guestrin, “XGBoost: A scalable tree boosting system,” in Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 2016, pp. 785–794.
Y. Wang, J. Zhang, and H. Li, “Comprehensive evaluation of adversarial perturbations against ML-based ethereum phishing detection systems,” ACM Transactions on the Web, 2025.
P. Yang, G. Zhao, and P. Zeng, “Phishing website detection based on multidimensional features driven by deep learning,” IEEE Access, vol. 7, pp. 15 196–15 209, 2019.
X. Zhang, J. Zhao, and Y. LeCun, “Character-level convolutional networks for text classification,” in Advances in Neural Information Processing Systems (NeurIPS), vol. 28, 2015, pp. 649–657.
S. Hochreiter and J. Schmidhuber, “Long short-term memory,” Neural Computation, vol. 9, no. 8, pp. 1735–1780, 1997.
A. S. Bozkir, F. C. Dalgic, and M. Aydos, “GramBeddings: A new neural network for URL-based malicious web site detection through n-gram embeddings,” Computers & Security, vol. 124, p. 103016, 2023.
H. Le, Q. Pham, D. Sahoo, and S. C. Hoi, “URLNet: Learning a URL representation with deep learning for malicious URL detection,” https://arxiv.org/abs/1802.03162, 2018, arXiv:1802.03162.
D. P. Kingma and J. Ba, “Adam: A method for stochastic optimization,” in International Conference on Learning Representations (ICLR), 2015.
N. Srivastava, G. Hinton, A. Krizhevsky, I. Sutskever, and R. Salakhutdinov, “Dropout: A simple way to prevent neural networks from overfitting,” Journal of Machine Learning Research, vol. 15, pp. 1929–1958, 2014.
I. J. Goodfellow, J. Shlens, and C. Szegedy, “Explaining and harnessing adversarial examples,” in International Conference on Learning Representations (ICLR), 2015.
A. Madry, A. Makelov, L. Schmidt, D. Tsipras, and A. Vladu, “Towards deep learning models resistant to adversarial attacks,” in International Conference on Learning Representations (ICLR), 2018.
N.CarliniandD.Wagner,“Towardsevaluatingtherobustnessof neural networks,” in IEEE Symposium on Security and Privacy (S&P), 2017, pp. 39–57.
B. Liu and G. Apruzzese, “Attacking logo-based phishing website detectors with adversarial perturbations,” arXiv preprint arXiv:2308.09392, 2023.
Y. Ji, H. Wang, and D. He, “Evaluating the effectiveness and robustness of visual similarity-based phishing detection models,” arXiv preprint arXiv:2405.19598, 2024.
J. Gao, J. Lanchantin, M. L. Soffa, and Y. Qi, “Black-box generation of adversarial text sequences to evade deep learning classifiers,” in IEEE Security and Privacy Workshops (SPW), 2018, pp. 50–56.
D. Jin, Z. Jin, J. T. Zhou, and P. Szolovits, “Is BERT really robust? a strong baseline for natural language attack on text classification and entailment,” in AAAI Conference on Artificial Intelligence, 2020, pp. 8018–8025.
B. Geng and G. Apruzzese, “Raze to the ground: Query-efficient adversarial HTML attacks on machine-learning phishing webpage detectors,” https://arxiv.org/abs/2310.03166, 2023, arXiv:2310.03166.
A. Yao, W. Li, Q. Zhu, and G. Gou, “From ML to LLM: Evaluating the robustness of phishing web page detection models against adversarial attacks,” arXiv preprint arXiv:2407.20361, 2024.
P. Agten, W. Joosen, F. Piessens, and N. Nikiforakis, “Seven months’ worth of mistakes: A longitudinal study of typosquatting abuse,” in Network and Distributed System Security Symposium (NDSS), 2015.
J. Spooren, D. Preuveneers, and W. Joosen, “Bypassing detection of URL-based phishing attacks using generative adversarial deep neural networks,” in ACM Workshop on Artificial Intelligence and Security (AISec), 2020, pp. 53–64.
L. Yu, W. Zhang, J. Wang, and Y. Yu, “SeqGAN: Sequence generative adversarial nets with policy gradient,” in AAAI Conference on Artificial Intelligence, 2017, pp. 2852–2858.
I. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. Courville, and Y. Bengio, “Generative adversarial nets,” in Advances in Neural Information Processing Systems (NeurIPS), 2014, pp. 2672–2680.
E. Gamma, R. Helm, R. Johnson, and J. Vlissides, Design Patterns: Elements of Reusable Ob ject-Oriented Software. Addison-Wesley, 1994.
S. Marchal, J. Francois, R. State, and T. Engel, “PhishStorm: Detecting phishing with streaming analytics,” IEEE Transactions on Network and Service Management, vol. 11, no. 4, pp. 458–471, 2014.
A. Costello, “RFC 3492: Punycode: A bootstring encoding of Unicode for internationalized domain names in applications (IDNA),” IETF. https://www.rfc-editor.org/rfc/rfc3492, 2003.
T. Bai, J. Luo, J. Zhao, B. Wen, and Q. Wang, “Recent advances in adversarial training for adversarial robustness,” arXiv preprint arXiv:2102.01356, 2021.
Kornjuhina, S. P., and O. R. Laponina. "Issledovanie vozmozhnostej algoritmov glubokogo obuchenija dlja zashhity ot fishingovyh atak." International Journal of Open Information Technologies 11.6 (2023): 163-174.
Namiot, D. E. Shemy atak na modeli mashinnogo obuchenija / D. E. Namiot // International Journal of Open Information Technologies. – 2023. – T. 11, # 5. – S. 68-86. – EDN YVRDOB.
Namiot, D. E. Osvedomlennost' o fishinge - voprosy obuchenija / D. E. Namiot, V. A. Vasenin // Sovremennye informacionnye tehnologii i IT-obrazovanie. – 2025. – T. 21, # 2. – S. 221-229. – DOI 10.25559/SITITO.021.202502.221-229. – EDN SLPOSM.
Refbacks
- There are currently no refbacks.
Abava Кибербезопасность Monetec 2026 СНЭ
ISSN: 2307-8162