Finite mixture of factorization machines

Abstract

This mini-dissertation will introduce a novel mixture model of factorization machines. Factorization machines (FM) are a supervised learning class capable of learning pairwise interactions between response variables which can also be extended to interactions in higher dimensions. They are based on matrix factorization techniques attributing to their success in prediction tasks. The FM factorizes interaction terms, obtaining prediction accuracy on par with multiple linear regression (MLR). The FM also achieves this using less variables, and the model performance exceeds MLR under sparsity. Finite Gaussian mixture models (FGMM) are adept at modeling non-homogeneous populations and detecting subgroups; however, they are constructed as a combination of multiple Gaussian linear regression components. The novel model will be constructed using a combination of multiple Gaussian factorization machines to exploit the advantages of FMs when it comes to pairwise interaction terms and sparsity. The model will be estimated in an expectation-maximization (EM) algorithm setting using a coordinate descent (CD) method to estimate the FM model equation. Compared to FGMM in a sparse data setting, the novel model achieves a better fit to the data using fewer parameters and a shorter computation time.