Proactive Caching for Vehicular Multi View 3D Video Streaming via Deep Reinforcement Learning in NS2

Abstract:

This paper investigates the problem of proactive caching for multi-view 3D videos in the fifth generation (5G) networks. We establish a mathematical model for this problem, and point out that it is difficult to solve the problem with traditional dynamic programming, then we propose a deep reinforcement learning approach to solve it First, we model the proactive caching system for multi-view 3D videos as a Markov decision process jointing views selection and local memory allocation. Then we present an actor-critic, model-free algorithm based on the deep deterministic policy gradient to find effective proactive caching policy. Since the action space is affected by the system state, we embed dynamic k-Nearest Neighbor algorithm into actor-critic algorithm to implement the deep reinforcement learning algorithm working in a action space of variable size. Finally, numerical results are given to demonstrate that the proposed solution can effectively maintain high-quality user experience for high-mobility 5G users moving among small cells. We also investigate the impact of configuration of critical parameters on the performance of the algorithm.