One desired aspect of microservices architecture is the ability to self-adapt
its own architecture and behaviour in response to changes in the operational
environment. To achieve the desired high levels of self-adaptability, this
research implements the distributed microservices architectures model, as
informed by the MAPE-K model. The proposed architecture employs a multi
adaptation agents supported by a centralised controller, that can observe the
environment and execute a suitable adaptation action. The adaptation planning
is managed by a deep recurrent Q-network (DRQN). It is argued that such
integration between DRQN and MDP agents in a MAPE-K model offers distributed
microservice architecture with self-adaptability and high levels of
availability and scalability. Integrating DRQN into the adaptation process
improves the effectiveness of the adaptation and reduces any adaptation risks,
including resources over-provisioning and thrashing. The performance of DRQN is
evaluated against deep Q-learning and policy gradient algorithms including: i)
deep q-network (DQN), ii) dulling deep Q-network (DDQN), iii) a policy gradient
neural network (PGNN), and iv) deep deterministic policy gradient (DDPG). The
DRQN implementation in this paper manages to outperform the above mentioned
algorithms in terms of total reward, less adaptation time, lower error rates,
plus faster convergence and training times. We strongly believe that DRQN is
more suitable for driving the adaptation in distributed services-oriented
architecture and offers better performance than other dynamic decision-making
algorithms.