Enhancing Cloud Task Scheduling with Multi-Objective Optimization Using K-Means Clustering and Dynamic Resource Allocation

The scheduling and resource allocation procedure is an essential component of cloud resource management. Effective resource allocation is severely hampered by the task arrival rates' erratic and unclear behavior. To prevent under or overusing resources, an effective scheduling strategy is necessary. To improve scheduling and allocation performance, a multi-objective optimization technique is presented for the best resource allocation and task scheduling inside scientific workflow datasets in a heterogeneous environment. In the first stage, the system calculates four key metrics: Communication Cost, Computation Cost, Earliest Finished Time on a particular VM, and Total Task Length for a specific scientific workflow dataset. These metrics provide a comprehensive understanding of the resource requirements and help make informed scheduling decisions. In the second stage, tasks are clustered using the K-Means clustering algorithm. This clustering groups similar tasks together, making managing and scheduling them easier. In the third stage, the proposed resource allocation algorithm allocates the clustered tasks to the appropriate VMs. This step ensures that the tasks are assigned to the best-suited resources, optimizing the overall system performance and resource utilization. By following this multi-stage process, the system aims to achieve optimal resource allocation and task scheduling, thereby improving the efficiency and effectiveness of cloud resource management. The proposed method significantly outperforms PSO, CSO, and GWO by consistently achieving lower Makespan—under 400 units at 50 tasks—while maintaining high resource utilization rates above 0.75, demonstrating superior efficiency in task execution and resource management.