Randomized Smoothing in Certified Robustness: Theory and a Systematic Review

Karine Ayrapetyants, Eugene Ilyushin

Abstract


Nowadays, as artificial intelligence systems are increasingly applied across various domains, the issue of their security has become ever more relevant. Naturally, neural network algorithms, which we currently associate with the concept of “artificial intelligence,” are also susceptible to both intentional and unintentional perturbations. Therefore, providing guarantees for the robustness of their operation is an important task. One of the methods that enables addressing this problem is randomized smoothing. This method allows us to obtain formal guarantees on the performance of a classifier under a given data distribution. Randomized smoothing, as well as its modifications, will be reviewed in this survey.

Full Text:

PDF (Russian)

References


J. M. Cohen, E. Rosenfeld и J. Z. Kolter, Certified Adversarial Robustness via Randomized Smoothing, 2019.

V. Voracek и M. Hein, «Improving l1-Certified Robustness via Randomized Smoothing by Leveraging Box Constraints,» в Proceedings of the 40th International Conference on Machine Learning, A. Krause, E. Brunskill, K. Cho, B. Engelhardt, S. Sabato и J. Scarlett, ред., сер. Proceedings of Machine Learning Research, т. 202, PMLR, 23–29 Jul 2023, с. 35 198—35 222. url: https://proceedings.mlr.press/v202/voracek23a.html

Y. Carmon, A. Raghunathan, L. Schmidt, P. Liang и J. C. Duchi, Unlabeled Data Improves Adversarial Robustness, 2022.

H. Zhang, Y. Yu, J. Jiao, E. P. Xing, L. E. Ghaoui и M. I. Jordan, Theoretically Principled Trade-off between Robustness and Accuracy, 2019. url: https://arxiv.org/abs/1901.08573

L. Schmidt, S. Santurkar, D. Tsipras, K. Talwar и A. Mądry, Adversarially Robust Generalization Requires More Data, 2018. url: https: //arxiv.org/abs/1804.11285

A. Krizhevsky и G. Hinton, «Learning multiple layers of features from tiny images,» University of Toronto, тех. отч. TR-2009, 2009. url: https://www.cs.toronto. edu/~kriz/learning-features-2009-TR.pdf

H. Salman и др., Provably Robust Deep Learning via Adversarially Trained Smoothed Classifiers, 2020.

A. Madry, A. Makelov, L. Schmidt, D. Tsipras и A. Vladu, Towards Deep Learning Models Resistant to Adversarial Attacks, 2019. url: https://arxiv.org/abs/ 1706.06083

J. Rony, L. G. Hafemann, L. S. Oliveira, I. B. Ayed, R. Sabourin и E. Granger, Decoupling Direction and Norm for Efficient Gradient-Based L2 Adversarial Attacks and Defenses, 2019. url: https://arxiv.org / abs/1811.09600

R. Zhai и др., MACER: Attack-free and Scalable Robust Training via Maximizing Certified Radius, 2022.

J. Jeong, S. Park, M. Kim, H.-C. Lee, D. Kim и J. Shin, SmoothMix: Training Confidence-calibrated Smoothed Classifiers for Certified Robustness, 2021.

J. Jeong и J. Shin, Consistency Regularization for Certified Robustness of Smoothed Classifiers, 2021.

L. Deng, «The mnist database of handwritten digit images for machine learning research,» IEEE Signal Processing Magazine, т. 29, № 6, с. 141—142, 2012.

J. Jeong, S. Kim и J. Shin, Confidence-aware Training of Smoothed Classifiers for Certified Robustness, 2022.

Z. Shi, Y. Wang, H. Zhang, J. Yi и C.-J. Hsieh, Fast Certified Robust Training with Short Warmup, 2021.

S. Gowal и др., On the Effectiveness of Interval Bound Propagation for Training Verifiably Robust Models, 2019.

V. Nair и G. E. Hinton, «Rectified linear units improve restricted boltzmann machines,» в Proceedings of the 27th International Conference on International Conference on Machine Learning, сер. ICML’10, Haifa, Israel: Omnipress, 2010, с. 807—814.

J. Jia, X. Cao, B. Wang и N. Z. Gong, Certified Robustness for Top-k Predictions against Adversarial Perturbations via Randomized Smoothing, 2019.

H. Salman, M. Sun, G. Yang, A. Kapoor и J. Z. Kolter, Denoised Smoothing: A Provable Defense for Pretrained Classifiers, 2020.

N. Carlini, F. Tramer, K. D. Dvijotham, L. Rice, M. Sun и J. Z. Kolter, (Certified!!) Adversarial Robustness for Free! 2023.

C. Xiao и др., DensePure: Understanding Diffusion Models towards Adversarial Robustness, 2022.

J. Jeong и J. Shin, Multi-scale Diffusion Denoised Smoothing, 2023.

H. Chen и др., Your Diffusion Model is Secretly a Certifiably Robust Classifier, 2024.

L. Li и др., TSS: Transformation-Specific Smoothing for Robustness Certification, 2021.

M. Fischer, M. Baader и M. Vechev, Certified Defense to Image Transformations via Randomized Smoothing, 2021.

B. Li, C. Chen, W. Wang и L. Carin, Certified Adversarial Robustness with Additive Noise, 2019.

M. Fischer, M. Baader и M. Vechev, Scalable Certified Segmentation via Randomized Smoothing, 2022.

O. Laousy и др., Towards Better Certified Segmentation via Diffusion Models, 2023. url: https://arxiv.org/abs/2306.09949

M. Weber, X. Xu, B. Karlaš, C. Zhang и B. Li, RAB: Provable Robustness Against Backdoor Attacks, 2023.

Z. Hammoudeh и D. Lowd, Provable Robustness Against a Union of ℓ0 Adversarial Attacks, 2024.

M. Lecuyer, V. Atlidakis, R. Geambasu, D. Hsu и S. Jana, Certified Robustness to Adversarial Examples with Differential Privacy, 2019.

C. Dwork, «Differential Privacy: A Survey of Results,» в Theory and Applications of Models of Computation, M. Agrawal, D. Du, Z. Duan и A. Li, ред., Berlin, Heidelberg: Springer Berlin Heidelberg, 2008, с. 1—19.

B.-H. Kung и S.-T. Chen, Towards Large Certified Radius in Randomized Smoothing using Quasiconcave Optimization, 2023. url: https://arxiv.org/abs/2302. 00209

S. Xia, Y. Yu, X. Jiang и H. Ding, Mitigating the Curse of Dimensionality for Certified Robustness via Dual Randomized Smoothing, 2024. url: https://arxiv. org/abs/2404.09586

L. Ding, T. Hu, J. Jiang, D. Li, W. Wang и Y. Yao, Random Smoothing Regularization in Kernel Gradient Descent Learning, 2023. url: https://arxiv.org/abs/2305.03531

S. Pfrommer, B. G. Anderson и S. Sojoudi, Projected Randomized Smoothing for Certified Adversarial Robustness, 2023. url: https://arxiv.org /abs/2309.13794

R. Hase, Y. Wang, T. Koike-Akino, J. Liu и K. Parsons, Variational Randomized Smoothing for Sample-Wise Adversarial Robustness, 2024. url: https://arxiv.org/abs/2407.11844

V. Rostermundt и B. G. Anderson, «Certified Adversarial Robustness via Mixture-of-Gaussians Randomized Smoothing,» в NeurIPS 2025 Workshop: Reliable ML from Unreliable Data, 2025. url: https://openreview.net/forum?id=ZyGMTcNaio

H. Hong, A. Kundu, A. Payani, B. Wang и Y. Hong, Towards Strong Certified Defense with Universal Asymmetric Randomization, 2025. url: https : / /arxiv. org/abs/2510.19977

V. Voracek, Treatment of Statistical Estimation Problems in Randomized Smoothing for Adversarial Robustness, 2025. url: https : / / arxiv . org / abs / 2406 . 17830

E. Seferis, C. Wu, S. Kollias, S. Bensalem и C.-H. Cheng, Randomized Smoothing Meets Vision- Language Models, 2025. url: https : / /arxiv.org/ abs / 2509.16088

E. Seferis, «Scaling Randomized Smoothing to state- of-the-art Vision-Language Models,» в ICLR 2025 Workshop: VerifAI: AI Verification in the Wild, 2025. url: https://openreview.net/forum?id=hyZePf0jxy

G. Yang, T. Duan, J. E. Hu, H. Salman, I. Razenshteyn и J. Li, Randomized Smoothing of All Shapes and Sizes, 2020.

C. Liang и X. Wu, Mist: Towards Improved Adversarial Examples for Diffusion Models, 2023.

A. Pal и J. Sulam, Understanding Noise-Augmented Training for Randomized Smoothing, 2023.

S. Wu, J. Wang, W. Ping, W. Nie и C. Xiao, Defending against Adversarial Audio via Diffusion Model, 2023.

J. Buckman, A. Roy, C. Raffel и I. Goodfellow, «Thermometer Encoding: One Hot Way To Resist Adversarial Examples,» 2018. url: https://openreview.net/pdf?id=S18Su--CW

J. Deng, W. Dong, R. Socher, L.-J. Li, K. Li и L. Fei-Fei, «Imagenet: A large-scale hierarchical image database,» в 2009 IEEE conference on computer vision and pattern recognition, Ieee, 2009, с. 248— 255.

Y. Netzer, T. Wang, A. Coates, A. Bissacco, B. Wu и A. Ng, «Reading Digits in Natural Images with Unsupervised Feature Learning,» 2011. url: https://api.semanticscholar.org/CorpusID:16852518

K. He, X. Zhang, S. Ren и J. Sun, «Deep residual learning for image recognition,» в Proceedings of the IEEE conference on computer vision and pattern recognition, 2016, с. 770—778.

S. Zagoruyko и N. Komodakis, Wide Residual Networks, 2017. url: https://arxiv.org/abs/1605.07146

S. Xie, R. Girshick, P. Dollár, Z. Tu и K. He, Aggregated Residual Transformations for Deep Neural Networks, 2017. url: https://arxiv.org/abs/1611.05431

Y. LeCun, L. Bottou, Y. Bengio и P. Haffner, «Gradient-based learning applied to document recognition,» Proceedings of the IEEE, т. 86, № 11, с. 2278—2324, 1998.

A. Dosovitskiy и др., An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale, 2021. url: https://arxiv.org/abs/2010.11929

Y. Le и X. S. Yang, «Tiny ImageNet Visual Recognition Challenge,» 2015. url: https://api. semanticscholar.org/CorpusID:16664790

A. Radford и др., Learning Transferable Visual Models From Natural Language Supervision, 2021. url: https://arxiv.org/abs/2103.00020

J. Wang и др., Deep High-Resolution Representation Learning for Visual Recognition, 2020. url: https://arxiv.org/abs/1908.07919

A. X. Chang и др., ShapeNet: An Information-Rich 3D Model Repository, 2015. url: https://arxiv.org/abs/ 1512.03012

H. Bao, L. Dong и F. Wei, «BEiT: BERT Pre-Training of Image Transformers,» CoRR, т. abs/2106.08254, 2021. url: https://arxiv.org/abs/2106.08254

P. Münch, R. Mreches, M. Binder, H. A. Gündüz, X.-Y. To и A. McHardy, deepG: Deep Learning for Genome Sequence Data, R package version 0.3.1, https://deepg.de/, 2024. url: https : / / github . com / GenomeNet/deepG

G. Singh, T. Gehr, M. Püschel и M. Vechev,

«An abstract domain for certifying neural networks,» Proceedings of the ACM on Programming Languages, т. 3, с. 1—30, янв. 2019. doi: 10.1145/3290354

K. Pei, Y. Cao, J. Yang и S. Jana, «Towards Practical Verification of Machine Learning: The Case of Computer Vision Systems,» дек. 2017. doi: 10 . 48550/arXiv.1712.01785

J. Mohapatra, T.-W. Weng, P.-Y. Chen, S. Liu и L. Daniel, «Towards Verifying Robustness of Neural Networks Against A Family of Semantic Perturbations,» июнь 2020, с. 241—249. doi: 10 . 1109/CVPR42600.2020.00032

M. Fischer, M. Baader и M. T. Vechev, «Certification of Semantic Perturbations via Randomized Smoothing,» CoRR, т. abs/2002.12463, 2020. url: https://arxiv.org/abs/2002.12463

M. Cordts и др., «The Cityscapes Dataset for Semantic Urban Scene Understanding,» в Proc. of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2016.

J. Ho, A. Jain и P. Abbeel, «Denoising Diffusion Probabilistic Models,» CoRR, т. abs/2006.11239, 2020. url: https://arxiv.org/abs/2006.11239

J. Wang и др., «Deep High-Resolution Representation Learning for Visual Recognition,» CoRR, т. abs/1908.07919, 2019. url: http : / / arxiv . org / abs/1908.07919

Z. Chen и др., Vision Transformer Adapter for Dense Predictions, 2023. url: https : / / arxiv . org / abs / 2205 . 08534

GitHub - fastai/imagenette: A smaller subset of 10 easily classified classes from Imagenet, and a little more French — github.com, https://github.com/fastai/ imagenette.

Weather Dataset — kaggle.com, https://www.kaggle. com/datasets/muthuj7/weather-dataset.

Ames Housing Dataset — kaggle.com, https://www. kaggle . com / datasets / shashanknecrothapa / ames - housing-dataset.

B. Delattre, P. Caillon, Q. Barthélemy, E. Fagnou и A. Allauzen, Bridging the Theoretical Gap in Randomized Smoothing, 2025. url: https://arxiv.org/ abs/2504.02412


Refbacks

  • There are currently no refbacks.


Abava  Кибербезопасность ИТ конгресс СНЭ

ISSN: 2307-8162