Education, Science, Technology, Innovation and Life
Open Access
Sign In

The Design and Implementation of Image Parallel Processing Framework Based on Hadoop

Download as PDF

DOI: 10.23977/mcee2020.039

Author(s)

Shenkuo Wang, Shaofei Wu, Yiqi Jiang, Huajie Zhang, Ning Xia

Corresponding Author

Shaofei Wu

ABSTRACT

For the efficiency of image processing in traditional single-machine environment is low, and the image processing used Hadoop cluster has too large load of the main node. In this framework, the internal representation of images serialized in Hadoop and the image storage model named ImgBundleFile was designed, the OpenCV image processing library was introduced and the image processing interface using the Java development language was designed. In order to greatly shorten the image processing time of the framework, a MapTask load balancing strategy is proposed. Using framework for the face detection, experimental results show that the image parallel processing framework based on Hadoop has good stability and scalability, and the efficiency is significantly improved compared with the traditional single-machine environment.

KEYWORDS

Hadoop, MapReduce, Face Detection, Image Processing Framework

All published work is licensed under a Creative Commons Attribution 4.0 International License.

Copyright © 2016 - 2031 Clausius Scientific Press Inc. All Rights Reserved.