An SDN controller-based network slicing scheme using constrained reinforcement learning

Abstract

In order to meet the strong diversification of services that demand network flexibility that will be able to serve the dire need for transmission resources, network slicing was embraced as a plausible solution. Reinforcement learning (RL) has been applied in resource allocation (RA) problems, but has not yet marked the translation from traditional optimization approaches primarily due to its inability to satisfy state constraints. The aim of this article is to address this challenge. This article proposes a logical architecture for network slicing based on software-defined networking (SDN), where an SDN controller controls the network slicing process in a centralized fashion, and manages the resource allocation (RA) process with the help of the slice manager. The considered problem jointly addresses power and channel allocation using a hybrid access mode for ultra-reliable low-latency communication (URLLC) and enhanced mobile broadband (eMBB) slices. Proper assumptions on the arrival rates, packet length distributions, as well as power and delay constraints were used to design the behavior of the reward function to realize a constrained RL approach. Here, the Bellman optimality equation was reformulated into a primal-dual optimization problem through the use of Nesterov’s smoothing technique and the Legendre-Fenchel transformation. The proposed algorithm shows favorable performance over the traditional RL strategy in attributes favoring eMBB services, i.e., the average bit rate, and significantly outperforms both baselines in attributes favoring URLLC services, i.e., average latency. Systematically, on the power-delay performance evaluation, it shows that it can adapt very well in rapidly time-varying non-Markovian environments and still successfully satisfy the delay constraints of the applications hosted on a slice.