Variational Bayes approximation for inverse regression problems

Abstract

Inverse regression is a tool to predict an unknown explanatory variable for given observations of a response variable in a regression problem. The prediction problem is usually carried out in two stages: firstly, to fit the model relationship between the variables, and secondly, to predict the unknown explanatory variable. Both the problems, model fitting and prediction involve considerable computational burden. Previous work on the Bayesian approach to the problem have used MCMC, INLA and other numerical methods. This thesis aims to present an alternative fast variational Bayes (VB) approximation to Bayesian inference for inverse regression problems which claims to avoid the limitations of previous work. The VB method assumes independence between the parameters in the posterior distribution, thus provides fast approximations to Bayesian estimation problems. In contrast to INLA, it can be applied to models with many unknown parameters. In the thesis, the VB method is applied to a wider class of inverse regression problems classified into two classes: inverse latent regression and inverse non-latent regression which present challenges for the method’s accuracy and tractability. The VB method itself is not without limitations. Quick VB solutions are obtained at the cost of some loss of accuracy. Also, tractable application of the method is limited to conjugate- exponential (CE) models. It is attempted to increase the accuracy and tractability of the method outside CE models with the use of further approximations, such as a Gaussian approximation.