Michael Wellman, University of Michigan

Abstract

Evaluating deep multi-agent training algorithms (MATAs) can be quite complicated due to stochasticity in training and sensitivity of agent performance to the behavior of other agents. We propose a meta-game evaluation framework where each MATA is cast as a meta-strategy, and repeatedly sampling normal-form empirical games over combinations of meta-strategies resulting from various random seeds. These empirical games provide the basis for constructing a sampling distribution, using bootstrapping, over a variety of game analysis statistics. We suggest that this framework provides a principled approach for analyzing advanced AI methods that interact with humans or other AI actors.

[Video]

Speaker bio: Michael P. Wellman is Professor and Division Chair of Computer Science & Engineering at the University of Michigan. He received a PhD from the Massachusetts Institute of Technology in 1988 for his work in qualitative probabilistic reasoning and decision-theoretic planning. From 1988 to 1992, Wellman conducted research in these areas at the USAF’s Wright Laboratory. For the past 30+ years, his research has focused on computational market mechanisms and game-theoretic reasoning methods, with applications in electronic commerce, finance, and cyber-security. As Chief Market Technologist for TradingDynamics, Inc., he designed configurable auction technology for dynamic business-to-business commerce. Wellman previously served as Chair of the ACM Special Interest Group on Electronic Commerce (SIGecom), and as Executive Editor of the Journal of Artificial Intelligence Research. He is a Fellow of the Association for the Advancement of Artificial Intelligence and the Association for Computing Machinery.