[Compstats] Talk Wed 4pm: Bayesian Language Modelling

Pascal Poupart ppoupart at cs.uwaterloo.ca
Tue Jan 26 02:36:09 EST 2010


As promised, I will give the second compstats seminar of the year this 
Wednesday @ 4pm.  I will talk about Bayesian methods for language 
modelling and some research in collaboration with Google to extract 
short synthesizing phrases to label clusters of documents in an 
unsupervised fashion.  Title and abstract follow.



Title: Bayesian Language Modelling

Speaker: Pascal Poupart
Date: Wednesday, Jan 27
Time: 4pm
Location: TBA

Recent years have seen an increased use of Bayesian techniques for 
language modelling due to their flexibility, their ability to model 
various syntactic and semantic sructure and the possibility to include 
prior knowledge.  In this talk, I will first review existing techniques 
for topic modelling such as probabilistic latent semantic analysis, 
latent Dirichlet allocation and hierarchical Dirichlet processes.  I 
will also review co-location techniques such as n-gram models and the 
hierarchical Pitman-Yor process.  I will then discuss how topic models 
and co-location models can be combined to yield powerful language models 
that simultaneously capture some elements of syntax and semantics.  I 
will show the benefits of such models to extract short synthesizing 
phrases to label clusters of documents in an unsupervised fashion. 

If time permits, I will also discuss some of the challenges to extend 
these models to deal with entities and relations that are often thoughts 
as the pillars of deep natural language understanding.  I may also 
discuss some of the challenges to learn these models in an online 
fashion by sequential monte carlo techniques.

This work was done in collaboration with Ting Liu (UW), Ruitong Huang 
(UW), Andy Chiu (Google) and Finnegan Southey (Google)
This work is supported financially by a Google Research Award

Pascal Poupart
Associate Professor
David R. Cheriton School of Computer Science
University of Waterloo
200 University Avenue West
Waterloo, Ontario
Canada N2L 3G1
Web: http://www.cs.uwaterloo.ca/~ppoupart
Email: ppoupart at cs.uwaterloo.ca 
Telephone: 1-519-888-4567x36239 
Fax: 1-519-885-1208

More information about the Compstats mailing list