Lectures on Language Technology and Learning

Lectures on Language Technology and Learning
Uppsala, May 15, 2013

The computational linguistics group at Uppsala University is delighted to invite you to an afternoon of public lectures on language technology and learning by leading experts in the field. The lectures will take place in Room 7-0042, English Park Campus, Uppsala University, on the 15th of May according to the schedule below. Attendance is free for anyone interested.

13.15-14.00    Hal Daumé III
Department of Computer Science, University of Maryland
Better! Faster! Stronger (theorems)! Learning to Balance Accuracy and Efficiency when Predicting Linguistic Structures
Viewed abstractly, many classic problems in natural language processing can be cast as trying to map a complex input (e.g., a sequence of words) to a complex output (e.g., a syntax tree or semantic graph). This task is challenging both because language is ambiguous (learning difficulties) and represented with discrete combinatorial structures (computational difficulties). I will discuss some approaches we’ve been developing to build learning algorithms that explicitly learn to trade-off accuracy and efficiency, applied to a variety of language processing phenomena. Moreover, I will show that in some cases, we can actually obtain a model that is faster and more accurate by exploiting smarter learning algorithms. And yes, those algorithms come with stronger theoretical guarantees too.

14.00-14.45    Hedvig Kjellström
Computer Vision and Active Perception Lab, KTH Royal Institute of Technology
Factorized Topic Models
I will present a modification to a latent topic model, which makes the model exploit supervision to produce a factorized representation of the observed data. The structured parameterization separately encodes variance that is shared between classes from variance that is private to each class by the introduction of a new prior over the topic space. The approach allows for a more efficient inference and provides an intuitive interpretation of the data in terms of an informative signal together with structured noise. The factorized representation is shown to enhance inference performance for image, text, and video classification. This work is done together with Cheng Zhang and Carl Henrik Ek at KTH.

14.45-15.15    Break

15.15-16.00    Anders Søgaard
Center for Language Technology, University of Copenhagen
Learning with Antagonistic Adversaries
Supervised NLP tools and on-line services are often used on text that is very different from the manually annotated text used during development. Many discriminative learning algorithms are sensitive to distributional shifts because highly indicative features may swamp other indicative features during training. We present some recent attempts to adress this – as two-player games where an adversary tries to prevent the learner from generalizing over data. We present two new adversarial learning algorithms with applications to document classification and part-of-speech tagging.

16.00-16.45    Ryan McDonald
Google Research
Adapting Syntactic Analyzers to Specific Tasks and Domains
After decades of development, syntactic analyzers are still typically trained to optimize implicit loss functions on newswire text. Of course, this is rarely the setting in which these analyzers are used. In practice, syntactic parsers are employed in tasks like information extraction and machine translation where only a subset of the analysis is potentially leveraged. Furthermore, they are often expected to analyze text from domains greatly divergent from newswire, e.g., queries, scientific literature and Web2.0 content. In this talk, I will discuss adapting syntactic analyzers using methods that do not change the underlying model type or even the feature representation. Instead, I will show that we can simply change how the parameters are set during learning. This includes augmenting standard implicit loss functions with task-specific loss functions, leveraging weak training signals from the target domain and making training robust to systematic changes in the distribution of features at test time. Experiments on domain adaptation for part-of-speech tagging and syntactic pre-reordering for machine translation will be presented.

http://stp.lingfil.uu.se/~nivre/ltl-lectures.html