November 2023

In the context of team Cornet’s seminars, Andrea Fox (LIA) will present his research work on Safe Reinforcement Learning for Video Admission Control, on December 8, 2023, at 11:35 in the meeting room. Abstract: Mobile video cameras have become a pervasive commodity and represent an important candidate source to enhance video analytic applications. Yet, while available in large quantities, the limitations of the edge computing infrastructure require the careful selection of which video flows to process at any point in time to maximize the amount of information extracted by deployed applications. In this paper, we present an admission control scheme for mobile video streams originating from different areas and dispatched to multiple processing servers over an edge computing infrastructure. We introduce a model rooted in the theory of Constrained Markov Decision Processes (CMDPs) that captures the problem of ensuring adequate area coverage to applications, while accounting for constraints of edge servers and access network capacity. On top of this model, we develop two new policies based on specialized primal-dual constrained Reinforcement Learning methods that solve the optimal admission control problem. The first, called DR-CPO, adopts reward decomposition reinforcement learning. This technique effectively mitigates state-space explosion, achieves optimality, and significantly accelerates Plus d'infos

In the context of team Cornet’s seminars, Olivier Bilenne (LIA) will present his research work on Implementing fictitious play in partially observable stochastic games, on November 24, 2023, 11:35 in the meeting room. Abstract: Extensions of fictitious play to stochastic games have been recently examined in combination with reinforcement learning techniques inherent to Markov decision processes. We revisit this approach in the context of partially observable stochastic games. For this, we consider a two-player (finite-state) zero-sum stochastic game where one player (the attacker) has full visibility of the system, whereas the other player (the defender) has no access to the state of the opponent and must instead compose with public sources of information (in our setting: the actions played and their associated payoffs). We study a fictitious play dynamics where the players best response to the estimated empirical frequencies of action of their opponent. This sequence of play requires from the players to form beliefs on both their opponent’s strategy and on their own continuation payoff (modeled by a Q-function), based on the (full or partial) information that is available to them. The strategy estimation scheme, in particular, features a correction mechanism making up for delayed symptoms in the partially Plus d'infos