Date 
Topics 
Readings 

Introduction 

M 8/30 
Introduction Overview, applications, history 
Bishop 1 

W 9/1 
Math Review Calculus, linear algebra, optimization 
Linear algebra Bishop Appendix C 

M 9/6 
Labor Day No Class 

W 9/8 
Probability Probability, stats 
You do NOT need to read all Probability: Bishop Chapter 2 Bishop Appendix B 

M 9/13 
Machine Learning Foundations Learning definitions, settings 
Bishop 1 

W 9/15 
Decision Trees Construction, pruining, overfitting 

Supervised Learning: Linear Methods 

M 9/20 
Regression Least squares and regression 
Bishop 3 

W 9/22 
Classification Logistic Regression 
Bishop 4 

M 9/27 
Generative vs. discriminative Naive Bayes and Logistic Regression 

W 9/29 
Online methods Perceptron, multiclass, structured 
Blum. OnLine Algorithms in Machine Learning. 1996 Bishop 4.1.2 Reducing Multiclass to Binary (Sections 1, 2, 3) 

Supervised Learning: NonLinear Methods 

M 10/4 
Support Vector Machines Maxmargin classification and optimization 
Bishop 7.1 Bishop Appendix E 

W 10/6 
Kernel Methods Dual optmization, kernel trick 
Bishop 6.1, 6.2 

Tue 10/12 
Instance based learning Nearestneighbors 
Bishop 2.5 Mitchell 88.4 

W 10/13 
Neural Networks Neural Network models 
Bishop 5.1,5.2,5.3,5.5 

M 10/18 
Ensemble Methods Boosting 
Bishop 14.1,14.2,14.3 

W 10/20 
Current Trends in Supervised Learning 

Unsupervised Learning 

M 10/25 
Clustering ExpectationMaximization and kmeans 
Bishop 9 

W 10/27 
EM and Clustering 1 Gaussian mixture models 
Bishop 9 

M 11/1 
EM and Clustering 2 The EM Algorithm 
Bishop 9 

Graphical Models 

W 11/3 
Graphical models 1 Bayesian networks and conditional independence 
Bishop 8.1, 8.2 

M 11/8 
Graphical models 2 MRFs and Exact inference 
Bishop 8.3, 8.4 

W 11/10 
Graphical models 3 Inference 
Bishop 8.3, 8.4 

M 11/15 
Sequential graphical models 1 Max Sum and Max Product 
Bishop 13.1,13.2 

W 11/17 
Sequential graphical models 2 HMMs and CRFs 

Other Topics 

M 11/22 
Dimensionality reduction PCA, probabilistic PCA, LDA 
Bishop 12.1,12.2,12.3 

W 11/24 
Thanksgiving Break No class 


M 11/29 
Guest Lecture: Current Trends in Unsupervised Learning Largescale nonparametric methods 

W 12/1 
Semisupervised learning Guest Lecture 

M 12/13 9am 
Final Poster Session Time Project presentations 

