Heterogeneous Data Resource Integration Based on Big Data Platform
Download as PDF
DOI: 10.23977/EMELS2020008
Author(s)
Qing Nie, Wenqiang Zhang, Tingting Zhang
Corresponding Author
Qing Nie
ABSTRACT
The heterogeneous resource integration system based on big data platform is designed to solve the problems of data resource storage and resource application in the media field for many years. By introducing the related technologies of the big data platform, this paper presents the overall design scheme based on Hadoop from the technical architecture and data architecture. Compared with the traditional data warehouse, the system has greatly improved its performance in terms of data storage capacity, data computing capacity and data query capability, which embodies the efficiency of big data platform in TB/PB data processing.
KEYWORDS
Multi-Source Heterogeneous Data; Hadoop; Distributed Storage; Distributed Storage; Mapreduce; Spark