Investigating a Dynamic Loop Scheduling with Reinforcement Learning Approach to Load Balancing in Scientific Applications

Rashid, M., Banicescu, I., & Carino, R.L. (2008). Investigating a Dynamic Loop Scheduling with Reinforcement Learning Approach to Load Balancing in Scientific Applications. Proc. 7th International Symposium on Parallel and Distributed Computing – ISPDC 2008. Krakow, Poland: IEEE Computer Society Press. 123-130.

The advantages of integrating reinforcement learning (RL) techniques into scientific parallel time-stepping applications have been revealed in research work over the past few years. In some of these previous works, the object of the integration is to automatically select the most appropriate dynamic loop scheduling (DLS) algorithm from a set of available algorithms with the purpose of improving the application performance via load balancing during the application execution. This paper investigates the performance of such a dynamic loop scheduling with reinforcement learning (DLS-with-RL) approach to load balancing. The DLS-with-RL is most suitable for use in timestepping scientific applications with parallel loops where the RL agent can learn application performance at the end of a time-step and take appropriate measures prior to starting the next time-step. The automatic selection is performed by the RL agent. The RL agent’s characteristics depend on a learning rate parameter and a discount factor parameter. In order to investigate the influences of these parameters, an application that simulates wavepacket dynamics is incorporated with a DLS-with-RL approach and allowed to execute on a cluster of workstations. The application contains three parallel loops with different time-varying characteristics. The RL agent implemented two RL algorithms: QLEARN and SARSA learning. Preliminary results indicate that on a fixed number of processors, the simulation completion time is not sensitive to the values of the learning parameters used in the experiments. The results also indicate that for this application, there is no advantage of choosing one RL technique over another, even though the techniques differed significantly in the number of times they selected the various DLS algorithms.