Abstract
The health context in which biomedicine and public health are operating is changing rapidly, such that we now routinely deal with both large n (sample size) and large p (dimension) datasets, and may also have multiple different types of data (transcriptomic, proteomic, metabolomic, ...) that we wish to integrate. Biomedical Bayesian analyses therefore require strategies for scaling both to large n and large p settings, motivating new inferential, modelling and/or computational approaches. This talk considers these challenges in the context of Bayesian mixture models for stratification analyses.
Motivated by the analysis of electronic health records (EHRs) for characterising patterns of multi-morbidity in UK populations, we present a divide-and-conquer approach for scaling Bayesian mixtures to model large-scale datasets. We review existing approaches, and consider the general challenge of matching cluster labels across data shards. We present a novel semi-heuristic technique for cluster matching and assessing cluster stability when Markov chain Monte Carlo (MCMC) techniques are used to perform inference, and introduce a recursive refinement approach to ensure that smaller clusters are not lost. To further enhance scalability and eliminate heuristic elements, we also present a novel variational inference procedure that incorporates (local) merge and delete moves to identify clusters within shards, followed by "global" merge moves across shards to identify overall clustering structures. We illustrate these techniques in the context of defining patient populations that have similar patterns of co-occurring long-term conditions.
If time permits, we will outline the additional challenges presented by the analysis of multiple, high-dimensional datasets, and briefly present our multiview and Bayesian correlated clustering approaches for dealing with these settings.