Introduction to Natural Language Processing (600.465)Mutual Information and Word Classes

The Problem

Word Classes

Solution

The New Model

Training Data

Training the New Model

Classes: How To Get Them

Creating the Word-to-Class Map

Simplifying the Objective Function

Maximizing Mutual Information(dependent on the mapping r)

Training or Heldout?

The Greedy Algorithm

Word Classes in Applications