Modelling Conditional Densities Using Finite Smooth Mixtures - Feng Li; Villani, Mattias; Kohn, Robert (Mixtures: Estimation and Applications, 2011)
Smooth mixtures, i.e. mixture models with covariate-dependent mixing weights, are very useful flexible models for conditional densities. Previous work shows that using too simple mixture components for modeling heteroscedastic and/or heavy tailed data can give a poor fit, even with a large number of components. This paper explores how well a smooth mixture of symmetric components can capture skewed data. Simulations and applications on real data show that including covariate-dependent skewness in the components can lead to substantially improved performance on skewed data, often using a much smaller number of components. Furthermore, variable selection is effective in removing unnecessary covariates in the skewness, which means that there is little loss in allowing for skewness in the components when the data are actually symmetric. We also introduce smooth mixtures of gamma and log-normal components to model positively-valued response variables.
Keywords: capable of approximating - large class of conditional distributions; finite smooth mixtures; first real dataset - using laser-emitted light to detect chemical compounds in atmosphere; generalised linear model (GLM) - to variable selection case; inference methodology - general MCMC scheme; LIDAR data; log predictive density score (LPDS); model comparison - components assumed known in MCMC scheme; modelling conditional densities - using finite smooth mixtures; or mixtures of experts (ME) - knowing machine learning literature; simple-and-many versus complex-and-few - modelling regression data-skewed response variable; simulation study in Villani et al. (2009) - smooth mixture of homoscedastic Gaussian components for heteroscedastic data; smooth mixtures