An agent-based workflow scheduling mechanism with deadline constraint on hybrid cloud environment

Abstract

With the advances of cloud computing, business and scientific-oriented jobs with certain workflows are increasingly migrated to and run on a variety of cloud environments. These jobs are often with the property of deadline constraint and have to be completed within limited time. Therefore, to schedule a job with workflow (short for workflow) with deadline constraint is increasingly becoming a crucial research issue. In this paper, we, based on previous work, propose an agent-based workflow scheduling mechanism to schedule workflows that are with deadline constraint into federated cloud environment. Design and MethodsWe add a workflow agent into the original framework to schedule the deadline-constraint workflow. The workflow agent can smoothly schedule workflows to the cloud system according to their required resource and automatically monitor their execution. In order to accurately predict the execution time of each task to meet deadline constraint on certain VM with given resource, we inherit the use of rough set theory to estimate execution time of task in our previous work. Result and DiscussionA heuristic algorithm that is embedded into the workflow agent is also proposed because the problem had been shown to be NP-complete. The mechanism also adopts dynamic job dispatching method to reduce the usage of VM and to improve the resource utilization. We also conducted experiments to evaluate the efficiency and effectiveness. ConclusionThe experimental results show that the prediction time is very close to the real execution time and can efficiently schedule multiple scientific workflows to meet the deadline constraints simultaneously.