Design and Implementation of Data Analysis System Based on Hadoop
Download as PDF
DOI: 10.23977/icmmct.2019.62104
Corresponding Author
Xiang Cui
ABSTRACT
With the increasing amounts of data, a single host can no longer meet the needs of computing and storage. At present, we mainly use distributed computing and storage methods to analyze and process large amounts of data, and tap potential value from it. Hadoop platform is the most widely used open source computing and storage framework. This paper analyses the functional requirements of data analysis system, and designs a data analysis system based on Hadoop, including data collection module, Hadoop module and HBase module. Experiment shows that, compared with the traditional database, the system has obvious advantages in dealing with massive data.
KEYWORDS
Data analysis system, Data mining, Hadoop