Statistics and Data Science Seminar - Adaptive Variational Bayes: Optimality, Computation and Applications

Abstract: In this talk, I will discuss adaptive statistical inference based on variational Bayes. Although a number of studies have been conducted to analyze theoretical properties such as posterior contraction properties of variational posteriors, there is still a lack of general and computationally tractable variational Bayes methods that can achieve  adaptive inference. To fill this gap, we propose a novel adaptive variational Bayes framework, which can operate on a collection of models.  The proposed framework first computes a variational posterior over each individual model separately and then combines them with certain weights to produce a variational posterior over the entire model space. It turns out that this combined variational posterior is the closest member to the posterior over the entire model in a predefined family of approximating distributions. We show that the proposed variational posterior achieves optimal contraction rates adaptively under very general conditions and attains model selection consistency when the true model structure exists. We apply the general results obtained for the adaptive variational Bayes to a large class of statistical models  including deep learning models and derive some new and adaptive inference results.

Biography: Lizhen Lin is a professor of statistics in the Department of Mathematics at the University of Maryland, where she currently also serves as the director of the statistics program. Her areas of expertise are in Bayesian modeling and theory for high-dimensional and infinite-dimensional models, geometry & statistics, statistical network analysis and statistical properties of deep generative models. 

Host: Debashis Mondal