Introduction to Natural Language Processing (600.465) Mutual Information and Word Classes

The Problem

Word Classes

Solution

The New Model

Training Data

Training the New Model

Classes: How To Get Them

Creating the Word-to-Class Map

Simplifying the Objective Function

Maximizing Mutual Information (dependent on the mapping r)

Training or Heldout?

The Greedy Algorithm

Word Classes in Applications