Introduction to Natural Language Processing (600.465)Maximum Entropy

Table of Contents

Introduction to Natural Language Processing (600.465)Maximum Entropy

Maximum Entropy??

The Maximum Entropy Principle

Example

Using Non-Maximum Entropy Distribution

Things in Perspective: n-gram LM

Features and Constraints

Additional Constraint (Ensuring Probability Distribution)

The Model

Loglinear (Exponential) Model

Getting the Lambdas: Setup

Generalized Iterative Scaling

Comments on Features

Feature Selection

References

Email: hajic@cs.jhu.edu

Home Page: http://www.cs.jhu.edu/~hajic/courses/cs465/syllabus.html