Plsa videos and info

Plate notation representing the PLSA model.

θ

is the document variable (

d

in the text),

z

is a topic (

c

in the text) drawn from the topic distribution for this document,

P (z | θ)

, and

w

is a word drawn from the word distribution for this topic,

P (w | z)

. The

d

and

w

are observable variables, the topic

z

is a latent variable.

Probabilistic latent semantic analysis (PLSA), also known as probabilistic latent semantic indexing (PLSI, especially in information retrieval circles) is a statistical technique for the analysis of two-mode and co-occurrence data. PLSA evolved from Latent semantic analysis, adding a sounder probabilistic model. PLSA has applications in information retrieval and filtering, natural language processing, machine learning from text, and related areas. It was introduced in 1999 by Jan Puzicha and Thomas Hofmann,^[1] and it is related to non-negative matrix factorization.

Considering observations in the form of co-occurrences

(w, d)

of words and documents, PLSA models the probability of each co-occurrence as a mixture of conditionally independent multinomial distributions:

The first formulation is the symmetric formulation, where

w

and

d

are both generated from the latent class

c

in similar ways (using the conditional probabilities

P (d | c)

and

P (w | c)

), whereas the second formulation is the asymmetric formulation, where, for each document

d

, a latent class is chosen conditionally to the document according to

P (c | d)

, and a word is then generated from that class according to

P (w | c)

. Although we have used words and documents in this example, the co-occurrence of any couple of discrete variables may be modelled in exactly the same way.

It is reported that the aspect model used in the probabilistic latent semantic analysis has severe overfitting problems^[2]. The number of parameters grows linearly with the number of documents. In addition, although PLSA is a generative model of the documents in the collection it is estimated on, it is not a generative model of new documents.

Evolutions of PLSA

References and notes

^ Thomas Hofmann, Probabilistic Latent Semantic Indexing, Proceedings of the Twenty-Second Annual International SIGIR Conference on Research and Development in Information Retrieval (SIGIR-99), 1999
^ Blei, David M.; Andrew Y. Ng, Michael I. Jordan (2003). "Latent Dirichlet Allocation". Journal of Machine Learning Research 3: 993–1022. doi:10.1162/jmlr.2003.3.4-5.993. http://jmlr.csail.mit.edu/papers/volume3/blei03a/blei03a.pdf.
^ Thomas Hofmann, Learning the Similarity of Documents : an information-geometric approach to document retrieval and categorization, Advances in Neural Information Processing Systems 12, pp-914-920, MIT Press, 2000
^ Alexei Vinokourov and Mark Girolami, A Probabilistic Framework for the Hierarchic Organisation and Classification of Document Collections, in Information Processing and Management, 2002
^ Eric Gaussier, Cyril Goutte, Kris Popat and Francine Chen, A Hierarchical Model for Clustering and Categorising Documents, in "Advances in Information Retrieval -- Proceedings of the 24th BCS-IRSG European Colloquium on IR Research (ECIR-02)", 2002

Diemazz

Contents

Evolutions of PLSA

References and notes

See also

External links