Author(s)

Suxing Hua ¹, Qiqi Gao ¹, Zhenling Wang ¹

Affiliation(s)

¹ Wuxi Institute of Technology, Wuxi, Jiangsu, 214000, China

Corresponding Author

Qiqi Gao

ABSTRACT

The rapid growth of data-intensive applications has led to an increased demand for efficient scheduling strategies in cloud computing environments. This paper focuses on the optimization of scheduling for data-intensive workflows, addressing the challenges posed by resource heterogeneity, complex workflow dependencies, and the need for scalability and elasticity. We propose a comprehensive approach that encompasses advanced task scheduling algorithms, dynamic resource allocation techniques, and effective parallelization and pipelining methods to enhance the performance of these workflows. The paper begins by characterizing data-intensive workflows and the cloud computing environment, highlighting the performance metrics crucial for evaluating workflow execution. It then delves into the scheduling challenges, discussing the implications of resource heterogeneity, the complexity of workflow dependencies, and the scalability and elasticity requirements of cloud-based workflows. We present optimization strategies that leverage heuristic and metaheuristic algorithms to schedule tasks efficiently, considering both task characteristics and resource capabilities. The resource allocation techniques discussed aim to optimize the utilization of cloud resources, adapting to the dynamic nature of the environment and the varying demands of tasks.

KEYWORDS

Cloud Computing; Data-Intensive Workflows; Scheduling Optimization; Resource Allocation

CITE THIS PAPER

Suxing Hua, Qiqi Gao, Zhenling Wang, Scheduling optimization strategy for data intensive workflows in cloud computing. Computing, Performance and Communication Systems (2024) Vol. 8: 99-105. DOI: http://dx.doi.org/10.23977/cpcs.2024.080112.

REFERENCES

[1] Roberto C, Oliver K, Diego C, et al. Data journeys: Explaining AI workflows through abstraction [J]. Semantic Web, 2024, 15 (4): 1057-1083.
[2] Gadaleta D. Automated Workflows for Data Curation and Machine Learning to Develop Quantitative Structure-Activity Relationships. [J]. Methods in molecular biology (Clifton, N.J.), 2025, 2834 115-130.
[3] Gierend K, Krüger F, Genehr S, et al. Provenance Information for Biomedical Data and Workflows: Scoping Review. [J]. Journal of medical Internet research, 2024, 26 e51297.
[4] Stow M S, Gibbons C B, Iii R C L, et al. Exploring Ion Mobility Mass Spectrometry Data File Conversions to Leverage Existing Tools and Enable New Workflows. [J]. Journal of the American Society for Mass Spectrometry, 2024,
[5] Haghani V, Goyal A, Zhang A, et al. Improving rigor and reproducibility in chromatin immunoprecipitation assay data analysis workflows with Rocketchip. [J]. bioRxiv : the preprint server for biology, 2024,
[6] Ding Y, Huang Y, Gao P, et al. Brain image data processing using collaborative data workflows on Texera [J]. Frontiers in Neural Circuits, 2024, (18) 1398884.
[7] Shrivastava M. Optimal Data Placement for Scientific Workflows in Cloud [J]. The Journal of Computer Information Systems, 2024, 64 (4): 501-517.
[8] Tholen D. Gasanalyze R: advancing reproducible research using a new R package for photosynthesis data workflows. [J]. AoB PLANTS, 2024, 16 (4): plae035.
[9] Florian R, Benjamin S, Kai E. Implementing Data Workflows and Data Model Extensions with RDF-star [J]. The Electronic Library, 2024, 42 (3): 393-412.
[10] Simon A, Ned L. Getting Started with DuckDB: A practical guide for accelerating your data science, data analytics, and data engineering workflows [M]. Packt Publishing Limited: 2024-06-24. DOI:10.0000/9781803232539.
[11] Lorella V. Editorial: Data and Workflows for Multilingual Digital Humanities [J]. Journal of Open Humanities Data, 2024, (10) 37.

Sponsors, Associates, and Links

---

• Power Systems Computation
  
  
• Internet of Things (IoT) and Engineering Applications
  
  
• Journal of Artificial Intelligence Practice
  
  
• Advances in Computer, Signals and Systems
  
  
• Journal of Network Computing and Applications
  
  
• Journal of Web Systems and Applications
  
  
• Journal of Electrotechnology, Electrical Engineering and Management
  
  
• Journal of Wireless Sensors and Sensor Networks
  
  
• Journal of Image Processing Theory and Applications
  
  
• Mobile Computing and Networking
  
  
• Vehicle Power and Propulsion
  
  
• Frontiers in Computer Vision and Pattern Recognition
  
  
• Knowledge Discovery and Data Mining Letters
  
  
• Big Data Analysis and Cloud Computing
  
  
• Electrical Insulation and Dielectrics
  
  
• Crypto and Information Security
  
  
• Journal of Neural Information Processing
  
  
• Collaborative and Social Computing
  
  
• International Journal of Network and Communication Technology
  
  
• File and Storage Technologies
  
  
• Frontiers in Genetic and Evolutionary Computation
  
  
• Optical Network Design and Modeling
  
  
• Journal of Virtual Reality and Artificial Intelligence
  
  
• Natural Language Processing and Speech Recognition
  
  
• Journal of High-Voltage
  
  
• Programming Languages and Operating Systems
  
  
• Visual Communications and Image Processing
  
  
• Journal of Systems Analysis and Integration
  
  
• Knowledge Representation and Automated Reasoning
  
  
• Review of Information Display Techniques
  
  
• Data and Knowledge Engineering
  
  
• Journal of Database Systems
  
  
• Journal of Cluster and Grid Computing
  
  
• Cloud and Service-Oriented Computing
  
  
• Journal of Networking, Architecture and Storage
  
  
• Journal of Software Engineering and Metrics
  
  
• Visualization Techniques
  
  
• Journal of Parallel and Distributed Processing
  
  
• Journal of Modeling, Analysis and Simulation
  
  
• Journal of Privacy, Trust and Security
  
  
• Journal of Cognitive Informatics and Cognitive Computing
  
  
• Lecture Notes on Wireless Networks and Communications
  
  
• International Journal of Computer and Communications Security
  
  
• Journal of Multimedia Techniques
  
  
• Automation and Machine Learning
  
  
• Computational Linguistics Letters
  
  
• Journal of Computer Architecture and Design
  
  
• Journal of Ubiquitous and Future Networks