The Design and Implementation of Image Parallel Processing Framework Based on Hadoop
Download as PDF
DOI: 10.23977/mcee2020.039
Author(s)
Shenkuo Wang, Shaofei Wu, Yiqi Jiang, Huajie Zhang, Ning Xia
Corresponding Author
Shaofei Wu
ABSTRACT
For the efficiency of image processing in traditional single-machine environment is low, and the image processing used Hadoop cluster has too large load of the main node. In this framework, the internal representation of images serialized in Hadoop and the image storage model named ImgBundleFile was designed, the OpenCV image processing library was introduced and the image processing interface using the Java development language was designed. In order to greatly shorten the image processing time of the framework, a MapTask load balancing strategy is proposed. Using framework for the face detection, experimental results show that the image parallel processing framework based on Hadoop has good stability and scalability, and the efficiency is significantly improved compared with the traditional single-machine environment.
KEYWORDS
Hadoop, MapReduce, Face Detection, Image Processing Framework