600.466
Information Retrieval and Web Agents (Spring 2008)

Instructor:

• David Yarowsky

TAs, Office Hours and Review Sections:

• Times and Locations
• Final Exam Review: Thursday, May 8, 3:30 PM, 3rd-Floor Shaffer
• Final Exam Review: Friday, May 9, 3:30 PM, 3rd-Floor Shaffer
• Office hours: Wednesday 1:30 PM, and before/after review sessions
• Final Exam: Saturday, May 10, 9-12 AM, Shaffer 100
• Topics on Final Exam

Syllabus:

• PDF version
• Postscript version

Textbooks:

• R. Baeza-Yates and B. Ribeiro-Neto, Modern Information Retrieval, Addison Wesley Longman, 1999.
• C. Wong. Web Client Programming. O'Reilly and Associates, 1997. (This web link should contain the complete text for the book, which is out-of-print)
• K. Sparck Jones and P. Willett. Readings in Information Retrieval. San Francisco: Morgan Kaufmann, 1997.
• W. Frakes and R. Baeza-Yates. Information Retrieval: Data Structures and Algorithms. Englewood Cliffs, N.J. : Prentice Hall, 1992.
• F. Cheong, Internet Agents: Spiders, Wanderers, Brokers, and Bots. Indianapolis, IN : New Riders, 1996.

Student Survey:

Assignments:

• Homework 1 - PDF (due February 21, 2008)
• Homework 2 - PDF (due March 14, 2008)
• Homework 3 - PDF (due April 10, 2008)
• Homework 3 - data files (due April 10, 2008)
• Final Project - PDF (due May 13, 2008)
• Homework 4 - PDF (due April 24, 2008)

Resources:

• Perl resources
• Perl Turorial
• IND (machine Learning package)
• Harvest User Manual (HTML mirror)
• Harvest User Manual (PS + PDF via Citeseer)
• HTTP - WWW protocols

Topics:

• Information Retrieval - Topics include a comprehensive study of current document retrieval models, mail/news routing and filtering, document clustering, automatic indexing, query expansion, relevance feedback, user modelling, information visualization and usage pattern analysis.
• Text Understanding - This segment of the course will focus on additional language processing steps for template filling and information extraction from retrieved documents, including reference resolution, sense tagging and summarization. Emphasis will be placed on recent, primarily statistical methods.
• Web Agents and WWW Applications - The final segment of the course will explore current issues in information retrieval and data mining on the World Wide Web. It will focus on case studies of web agents, spiders, robots and search engines, exploring both their practical implementation and the economic and legal issues surrounding their use. One of the hot technologies of the 21st century!