Education, Science, Technology, Innovation and Life
Open Access
Sign In

Storage Optimization of Condition Monitoring Big Data of Transmission Based on Cloud Platform

Download as PDF

DOI: 10.23977/iceccs.2018.006


Min Ji, Peng Fei J

Corresponding Author

Min Ji


Applying big data technology for improving the condition evaluation of power transmission and transforming equipment and solving its practical problems becomes a new challenge in power industry. For high reliable storage and rapid access of data, the data distribution strategy, data block size adjustment and the cluster network topology are studied based on hadoop. A multi-copy consistency Hash algorithm based on data correlation (CMCH) is proposed. The algorithm makes the relevant data gathering in the cluster and improves the data processing speed. Based on the CMCH algorithm and Map Reduce model, a multiple data sources map join query algorithm and multi-channel data fusion feature extraction algorithm are designed. The two algorithms are executed on our built clusters and the results show that the CMCH improves the efficiency of multiple data sources join query and multi-channel data fusion feature extraction, and the execution time is just 32% and 35% respectively comparing with standard Hadoop.


Big data, Power transmission and transformation equipment, Consistency Hash, Cloud computing

All published work is licensed under a Creative Commons Attribution 4.0 International License.

Copyright © 2016 - 2031 Clausius Scientific Press Inc. All Rights Reserved.