A Gamma-Poisson topic model for short text

Show simple item record

dc.contributor.advisor De Waal, Alta
dc.contributor.coadvisor De Villiers, Pieter
dc.contributor.postgraduate Mazarura, Jocelyn Rangarirai
dc.date.accessioned 2021-02-12T09:59:13Z
dc.date.available 2021-02-12T09:59:13Z
dc.date.created 2021
dc.date.issued 2020
dc.description Thesis (PhD (Mathematical Statistics))--University of Pretoria, 2020. en_ZA
dc.description.abstract Most topic models are constructed under the assumption that documents follow a multinomial distribution. The Poisson distribution is an alternative distribution to describe the probability of count data. For topic modelling, the Poisson distribution describes the number of occurrences of a word in documents of fixed length. The Poisson distribution has been successfully applied in text classification, but its application to topic modelling is not well documented, specifically in the context of a generative probabilistic model. Furthermore, the few Poisson topic models in literature are admixture models, making the assumption that a document is generated from a mixture of topics. In this study, we focus on short text. Many studies have shown that the simpler assumption of a mixture model fits short text better. With mixture models, as opposed to admixture models, the generative assumption is that a document is generated from a single topic. One topic model, which makes this one-topic-per-document assumption, is the Dirichlet-multinomial mixture model. The main contributions of this work are a new Gamma-Poisson mixture model, as well as a collapsed Gibbs sampler for the model. The benefit of the collapsed Gibbs sampler derivation is that the model is able to automatically select the number of topics contained in the corpus. The results show that the Gamma-Poisson mixture model performs better than the Dirichlet-multinomial mixture model at selecting the number of topics in labelled corpora. Furthermore, the Gamma-Poisson mixture produces better topic coherence scores than the Dirichlet-multinomial mixture model, thus making it a viable option for the challenging task of topic modelling of short text. The application of GPM was then extended to a further real-world task: that of distinguishing between semantically similar and dissimilar texts. The objective was to determine whether GPM could produce semantic representations that allow the user to determine the relevance of new, unseen documents to a corpus of interest. The challenge of addressing this problem in short text from small corpora was of key interest. Corpora of small size are not uncommon. For example, at the start of the Coronavirus pandemic limited research was available on the topic. Handling short text is not only challenging due to the sparsity of such text, but some corpora, such as chats between people, also tend to be noisy. The performance of GPM was compared to that of word2vec under these challenging conditions on labelled corpora. It was found that the GPM was able to produce better results based on accuracy, precision and recall in most cases. In addition, unlike word2vec, GPM was shown to be applicable on datasets that were unlabelled and a methodology for this was also presented. Finally, a relevance index metric was introduced. This relevance index translates the similarity distance between a corpus of interest and a test document to the probability of the test document to be semantically similar to the corpus of interest. en_ZA
dc.description.availability Unrestricted en_ZA
dc.description.degree PhD (Mathematical Statistics) en_ZA
dc.description.department Statistics en_ZA
dc.identifier.citation * en_ZA
dc.identifier.uri http://hdl.handle.net/2263/78519
dc.language.iso en en_ZA
dc.publisher University of Pretoria
dc.rights © 2019 University of Pretoria. All rights reserved. The copyright in this work vests in the University of Pretoria. No part of this work may be reproduced or transmitted in any form or by any means, without the prior written permission of the University of Pretoria.
dc.subject Topic modelling for short text en_ZA
dc.subject Gamma-poisson mixture
dc.subject mixture models
dc.subject topic modelling
dc.subject document similarity
dc.title A Gamma-Poisson topic model for short text en_ZA
dc.type Thesis en_ZA


Files in this item

This item appears in the following Collection(s)

Show simple item record