In the context of team Cornet’s seminars, Olivier Bilenne (LIA) will present his research work on Implementing fictitious play in partially observable stochastic games, on November 24, 2023, 11:35 in the meeting room.
Abstract: Extensions of fictitious play to stochastic games have been recently examined in combination with reinforcement learning techniques inherent to Markov decision processes. We revisit this approach in the context of partially observable stochastic games. For this, we consider a two-player (finite-state) zero-sum stochastic game where one player (the attacker) has full visibility of the system, whereas the other player (the defender) has no access to the state of the opponent and must instead compose with public sources of information (in our setting: the actions played and their associated payoffs). We study a fictitious play dynamics where the players best response to the estimated empirical frequencies of action of their opponent. This sequence of play requires from the players to form beliefs on both their opponent’s strategy and on their own continuation payoff (modeled by a Q-function), based on the (full or partial) information that is available to them. The strategy estimation scheme, in particular, features a correction mechanism making up for delayed symptoms in the partially observable setting, thus enabling the defender to better predict the attacker’s strategy. The solution strategies of play, which map belief states to mixed best response strategies, are specified by only a finite number of parameters that form a equilibrium point with respect to expected payoff.