In recent years, various companies started to shift their data services from traditional data centers onto cloud. One of the major motivations is to save operation costs with the aid of cloud elasticity. This paper discusses an emerging need from financial services to reduce idle servers retaining very few user connections, without disconnecting them from the server side. This paper considers this need as a bi-objective online load balancing problem. A neural network based scalable policy is designed to route user requests to varied numbers of servers for elasticity. An evolutionary multi-objective training framework is proposed to optimize the weights of the policy. Not only the new objective of idleness is reduced by over 130% more than traditional industrial solutions, but the original load balancing objective is slightly improved. Extensive simulations help reveal the detailed applicability of the proposed method to the emerging problem of reducing idleness in financial services.
Overview
Dependencies
System
ubuntu 20.04
python 3.8.12
cuda 11.0
Packages
torch==1.11.0
numpy
Training
python xxx_train_xxx.py
Testing
python xxx_test_xxx.py
Results
Please check it in the paper doc/MERL-LB-v1.pdf.
github zlaom/MERL-LB: SCIS-23 MERL-LB: A PyTorch Implementation (github.com)
