面向AI系统的攻击与防御方法研究

Research progress on attack and defense methods for AI systems

摘要:

近年来,随着计算机硬件算力的大幅提升和算法的快速发展,人工智能(artificial intelligence,AI)在图像识别、汽车自动驾驶、辅助医疗诊断等多个领域取得显著的优势。然而,在AI系统运行的每个环节都有可能遭受来自外部的安全威胁。在数据收集阶段,基于AI 的运算系统容易受到传感器欺骗攻击;在数据预处理阶段,智能模型容易受到数据缩放攻击; 在模型的训练和推理阶段,系统容易受到数据投毒攻击和对抗攻击。为了更好地应对AI系统的潜在威胁,首先回顾AI安全问题的挑战和最新的研究进展,以AI系统生命周期为依据, 分阶段阐述系统所面临的安全威胁以及应对策略。在此基础上,概述了AI安全的总体架构。 最后,讨论了未来AI系统所面临的挑战。

Abstract:

In recent years, with the significant improvement in computer hardware capabili ties and the rapid development of algorithms, artificial intelligence (AI) has achieved remark able advantages in fields such as image recognition, autonomous vehicle driving, and assisted medical diagnosis. However, every link in the operation of AI systems may be subject to external security threats. During the data collection stage, the AI-based computing system is vulnerable to sensor spoofing attacks; during data pre-processing stage, intelligent models are susceptible to data scaling attacks; and during the model training and inference stages, the system is prone to data poisoning attacks and adversarial attacks. To better address the potential threats to AI systems, this paper first reviewed the challenges and latest research progress in AI security issues. Based on the lifecycle of AI systems, it elaborated on the security threats faced by the system at each stage and the corresponding countermeasures. On this basis, an overall architecture for AI security was outlined. Finally, the challenges faced by future AI systems were discussed.