基于深度强化学习的高效联邦学习客户端选择方法

Efficient client selection method for federated learning based on deep reinforcement learning

摘要:

联邦学习允许边缘设备在不共享原始数据的情况下协作训练人工智能模型，因其在效率、隐私保护和可扩展性方面的优势而受到广泛关注。随着人工智能模型复杂性的提升和多轮模型聚合需求的增加，联邦学习面临显著的能量消耗问题。以往的研究很少考虑本地训练对客户端选择的影响，导致理想与实际节能之间存在巨大差距。鉴于边缘设备在能力和数据方面的异构性，研究了联合优化联邦学习中的客户端选择和本地训练的问题，以提高能量效率，其中涉及2个耦合优化变量。为此，提出了一种新的多智能体深度强化学习方法——全局感知独立近端策略优化（global-aware independent proximal policy optimization, GIPPO）方法，可以根据客户端的状态自适应地选择客户端进行训练并调整迭代次数，从而最小化整体能量消耗，提升模型性能。此外，GIPPO可以根据剩余资源动态调整奖励以适应动态环境。仿真和实际原型的实验结果显示，与基准方法相比，该方法能够节省高达778%的能量成本。

Abstract:

Federated learning allows edge devices to collaboratively train artificial intelligence (AI) models without sharing raw data, attracting widespread attention due to its advantages in efficiency, privacy protection, and scalability. However, as the complexity of AI models increases and the need for multiple rounds of model aggregation grows, federated learning faces significant energy consumption issues. Previous research has rarely considered the impact of local training on client selection, leading to a significant gap between ideal and actual energy savings. Given the heterogeneity of edge devices in terms of capabilities and data, this paper investigated the joint optimization of client selection and local training in federated learning to improve energy efficiency, which involves two coupled variables.To this end, a novel multi-agent deep reinforcement learning method, global-aware independent proximal policy optimization (GIPPO), was proposed. It can adaptively select clients for training and adjust the number of iterations based on the state of each client, aiming to minimize overall energy consumption while enhancing model performance. Additionally, by dynamically adjusting rewards based on remaining resources, GIPPO can adapt to dynamic environments. Simulation and real prototype experiments show that this method can achieve up to 778% energy cost savings compared to baseline methods.