Federated-Learning Incentive Mechanism for Object Detection in Power Inspection Images
Zhong Linlin1, Liu Keyu2
1. School of Electrical Engineering Southeast University Nanjing 210096 China; 2. SEU-Monash Joint Graduate School (Suzhou) Southeast University Suzhou 215123 China
Abstract：In recent years, the automated inspection approaches have been widely used in power system, which is safer and more efficient than traditional manual inspection. In order to effectively process the large number of power inspection images generated by automated inspection machines, such as unmanned aerial vehicles (UAVs), the deep learning-based object detection technology is commonly used to timely detect potential faults. However, in practical applications, power lines are widely distributed, and the collected inspection images are not easily integrated. In addition, due to data privacy and restrictions from relevant laws and regulations, these inspection images cannot be shared with or provided to third-party organizations. The data exist as “data islands” scattered among various power companies. In this situation, a single power company has a limited amount and variety of inspection image data, making it difficult to obtain a robust object detection model with good generalization performance through traditional data-driven deep learning algorithms. Federated learning (FL), as an emerging distributed technology, can be efficiently applied to visual tasks of power inspection by constructing a high-performance model through collaboration among participants while protecting data privacy. In traditional federated learning, the final distributed model to each participant is a global model with the same performance without considering the contribution differences by different participants, which means different participants receive the same model. However, in real scenarios, the quantity, quality, and cost of power inspection image data provided by different participants are really different, and their contributions to the training of global model are also different. If the participants in federal learning cannot receive fair and reasonable returns, it will cause some participants to be unwilling to participate in the federal training of object detection model for processing power inspection images, and cannot guarantee the number of participants in federal learning, resulting in the inability to continue federal training or poor training performance. To address these challenges and incentivize more participants to join federated learning for object detection of power inspection images, this paper proposes two federated incentive mechanisms based on model fairness and revenue fairness, respectively. The model fairness incentive mechanism is designed for the scenarios where all participants are data owners, and it involves evaluating contributions to obtain models with varying performance levels. In this incentive mechanism, we use an evaluation strategy based on marginal benefit, and compare the contribution measurement methods based on unilateral training, dynamic accuracy, and impact function. The revenue fairness incentive mechanism targets the scenarios involving both data owners and data demanders, wherein data owners receive corresponding benefits, and data demanders obtain high-performance models. In this incentive mechanism, the fairness of the federated learning is modeled by evaluating contributions, expected losses, and expected temporal losses. The above two incentive mechanisms are validated by the experiments on the homemade UAV power tower inspection image dataset. The three kinds of data heterogeneity are considered including feature skew, quantity skew and mixed skew. The results demonstrate that the fairness correlation coefficients in the two incentive mechanisms can reach 0.96 and 1, respectively. This indicates that the proposed incentive mechanisms can effectively enhance fairness and motivate more participants to engage in federated learning for object detection of power inspection images. Also, it is noted that although we use power object detection as the main application scenario in this work, the proposed incentive mechanism can be applied to various power vision tasks including image classification, object detection, and anomaly detection, and has a certain degree of generality.
仲林林, 刘柯妤. 面向电力巡检图像目标检测的联邦学习激励机制[J]. 电工技术学报, 2024, 39(17): 5434-5449.
Zhong Linlin, Liu Keyu. Federated-Learning Incentive Mechanism for Object Detection in Power Inspection Images. Transactions of China Electrotechnical Society, 2024, 39(17): 5434-5449.
[1] 郭敬东, 陈彬, 王仁书, 等. 基于YOLO的无人机电力线路杆塔巡检图像实时检测[J]. 中国电力, 2019, 52(7): 17-23. Guo Jingdong, Chen Bin, Wang Renshu, et al.YOLO-based real-time detection of power line poles from unmanned aerial vehicle inspection vision[J]. Electric Power, 2019, 52(7): 17-23. [2] 阎光伟, 马颐琳, 焦润海, 等. 航拍电力杆塔图像中销钉唯一性识别研究[J/OL]. 电工技术学报, 2023: 1-12[2023-09-11]. DOI:10.19595/j.cnki.1000-6753.tces. 230760. Yan Guangwei, Ma Yilin, Jiao Runhai, et al. Research on pin uniqueness identification in aerial power tower images[J/OL]. Transactions of China Electrotechnical Society, 2023: 1-12[2023-09-11]. DOI:10.19595/j.cnki. 1000-6753.tces.230760. [3] 宋立业, 刘帅, 王凯, 等. 基于改进EfficientDet的电网元件及缺陷识别方法[J]. 电工技术学报, 2022, 37(9): 2241-2251. Song Liye, Liu Shuai, Wang Kai, et al.Identification method of power grid components and defects based on improved EfficientDet[J]. Transactions of China Electrotechnical Society, 2022, 37(9): 2241-2251. [4] 苟军年, 杜愫愫, 刘力. 基于改进掩膜区域卷积神经网络的输电线路绝缘子自爆检测[J]. 电工技术学报, 2023, 38(1): 47-59. Gou Junnian, Du Susu, Liu Li.Transmission line insulator self-explosion detection based on improved mask region-convolutional neural network[J]. Transactions of China Electrotechnical Society, 2023, 38(1): 47-59. [5] 韩翔宇, 纽春萍, 何海龙, 等. 电磁式断路器状态监测与智能评估技术综述[J]. 电工技术学报, 2023, 38(8): 2191-2210. Han Xiangyu, Niu Chunping, He Hailong, et al.Review of condition monitoring and intelligent assessment of electromagnetic circuit breaker[J]. Transactions of China Electrotechnical Society, 2023, 38(8): 2191-2210. [6] Yang Qiang, Liu Yang, Chen Tianjian, et al.Federated machine learning: concept and applications[J]. ACM Transactions on Intelligent Systems and Technology, 2019, 10(2): 12. [7] 郭方洪, 刘师硕, 吴祥, 等. 基于联邦学习的含不平衡样本数据电力变压器故障诊断[J]. 电力系统自动化, 2023, 47(10): 145-152. Guo Fanghong, Liu Shishuo, Wu Xiang, et al.Federated learning based fault diagnosis of power transformer with unbalanced sample data[J]. Automation of Electric Power Systems, 2023, 47(10): 145-152. [8] Buttarelli G.The EU GDPR as a clarion call for a new global digital gold standard[J]. International Data Privacy Law, 2016, 6(2): 77-78. [9] Hard A, Rao K, Mathews R, et al. Federated learning for mobile keyboard prediction[J/OL]. ArXiv, 2019: 1811.03604v2. https://doi.org/10.48550/arXiv.1811.03604. [10] 梁天恺, 曾碧, 陈光. 联邦学习综述:概念、技术、应用与挑战[J]. 计算机应用, 2022, 42(12): 3651-3662. Liang Tiankai, Zeng Bi, Chen Guang.Federated learning survey: concepts, technologies, applications and challenges[J]. Journal of Computer Applications, 2022, 42(12): 3651-3662. [11] Ng K L, Chen Zichen, Liu Zelei, et al.A multi-player game for studying federated learning incentive schemes[C]//Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence, Yokohama, Japan, 2021: 5279-5281. [12] Wang Guan, Dang C X, Zhou Ziye.Measure contribution of participants in federated learning[C]//2019 IEEE International Conference on Big Data, Los Angeles, CA, USA, 2019: 2597-2604. [13] Richardson A, Filos-Ratsikas A, Faltings B. Rewarding high-quality data via influence functions [J/OL]. ArXiv, 2019: 1908. 11598. http://arxiv.org/abs/1908.11598v1. [14] Kang Jiawen, Xiong Zehui, Niyato D, et al.Incentive mechanism for reliable federated learning: a joint optimization approach to combining reputation and contract theory[J]. IEEE Internet of Things Journal, 2019, 6(6): 10700-10714. [15] Zhang Jingfeng, Li Cheng, Robles-Kelly A, et al.Hierarchically fair federated learning[J/OL]. ArXiv, 2020: 1-16. https://doi.org/10.48550/arXiv.2004.10386. [16] Lyu Lingjuan, Yu Jiangshan, Nandakumar K, et al.Towards fair and privacy-preserving federated deep models[J]. IEEE Transactions on Parallel and Distributed Systems, 2020, 31(11): 2524-2541. [17] Lyu Lingjuan, Xu Xinyi, Wang Qian, et al.Collaborative fairness in federated learning[M]//Yang Qiang, Fan Lixin, Yu Han. Federated Learning: Privacy and Incentive. Cham: Springer, 2020: 189-204. [18] Yu Han, Liu Zelei, Liu Yang, et al.A fairness-aware incentive scheme for federated learning[C]// Proceedings of the AAAI/ACM Conference on AI, Ethics, and Society, New York, NY, USA, 2020: 393-399. [19] Dwork C.Differential privacy: a survey of results[C]//The 5th International Conference on Theory and Applications of Models of Computation, Xi'an, China, 2008: 1-19. [20] Rivest R L, Adleman L, Dertouzos M L.On data banks and privacy homomorphisms[M]//DeMillo R A, Lipton R J, Dobkin D P, et al. Foundations of Secure Computation. Amstertam: Academic Press, 1978: 169-180. [21] Liu Keyu, Zhong Linlin.Object detection of UAV power line inspection images based on federated learning[C]//2022 IEEE 5th International Electrical and Energy Conference (CIEEC), Nanjing, China, 2022: 2372-2377. [22] Zeng Rongfei, Zeng Chao, Wang Xingwei, et al.A comprehensive survey of incentive mechanism for federated learning[J/OL]. ArXiv, 2021: 2106.15406v1. https://doi.org/10.48550/arXiv.2106.15406. [23] Zhan Yufeng, Zhang Jie, Hong Zicong, et al.A survey of incentive mechanism design for federated learning[J]. IEEE Transactions on Emerging Topics in Computing, 2022, 10(2): 1035-1044. [24] Tang Ming, Wong V W S. An incentive mechanism for cross-silo federated learning: a public goods perspective[C]//IEEE INFOCOM 2021-IEEE Conference on Computer Communications, Vancouver, BC, Canada, 2021: 1-10. [25] 梁文雅, 刘波, 林伟伟, 等. 联邦学习激励机制研究综述[J]. 计算机科学, 2022, 49(12): 46-52. Liang Wenya, Liu Bo, Lin Weiwei, et al.Survey of incentive mechanism for federated learning[J]. Computer Science, 2022, 49(12): 46-52. [26] 朱智韬, 司世景, 王健宗, 等. 联邦学习的公平性研究综述[J]. 大数据, 2024, 10(1): 62-85. Zhu Zhitao, Si Shijing, Wang Jianzong, et al.A survey on the fairness of federated learning[J]. Big Data Research, 2024, 10(1): 62-85. [27] Zhou Zirui, Chu Lingyang, Liu Changxin, et al.Towards fair federated learning[C]//Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining, Virtual Event, Singapore, 2021: 4100-4101. [28] McMahan H B, Moore E, Ramage D, et al. Communication-efficient learning of deep networks from decentralized data[J/OL]. ArXiv, 2016: 1602. 05629v4. https://doi.org/10.48550/arXiv.1602.05629. [29] 仲林林, 胡霞, 刘柯妤. 基于改进生成对抗网络的无人机电力杆塔巡检图像异常检测[J]. 电工技术学报, 2022, 37(9): 2230-2240, 2262. Zhong Linlin, Hu Xia, Liu Keyu.Power tower anomaly detection from unmanned aerial vehicles inspection images based on improved generative adversarial network[J]. Transactions of China Electrotechnical Society, 2022, 37(9): 2230-2240, 2262. [30] Li Qinbin, Diao Yiqun, Chen Quan, et al.Federated learning on non-IID data silos: an experimental study[C]//2022 IEEE 38th International Conference on Data Engineering (ICDE), Kuala Lumpur, Malaysia, 2022: 965-978. [31] Ratner B.The correlation coefficient: Its values range between +1/-1, or do they?[J]. Journal of Targeting, Measurement and Analysis for Marketing, 2009, 17(2): 139-142. [32] Kendall M G.Rank Correlation Methods[M]. London: Charles Griffin, 1948.
2024, Vol. 39 
