600.466
Information Retrieval and Web Agents (Spring 2009)

Instructor:

• David Yarowsky

TAs, Office Hours and Review Sections:

• Times and Locations
• Final Exam: Saturday, May 9, 2-5 PM
• Topics on Exam
• Course slides are available at /home/cs466/SLIDES on the undergraduate network

Syllabus:

• PDF version
• Postscript version

Textbooks:

• Manning, P. Raghavan and H. Schuetze, Introduction to Information Retrieval, Cambridge University Press, 2008. PRIMARY TEXT
• R. Baeza-Yates and B. Ribeiro-Neto, Modern Information Retrieval, Addison Wesley Longman, 1999.
• C. Wong. Web Client Programming. O'Reilly and Associates, 1997. (This web link should contain the complete text for the book, which is out-of-print)
• W. Frakes and R. Baeza-Yates. Information Retrieval: Data Structures and Algorithms. Englewood Cliffs, N.J. : Prentice Hall, 1992.
• F. Cheong, Internet Agents: Spiders, Wanderers, Brokers, and Bots. Indianapolis, IN : New Riders, 1996.

Student Survey:

Assignments:

• Homework 1 - PDF (due February 19, 2009)
• Homework 2 - PDF (due March 12, 2009)
• Homework 3 - PDF (due April 2, 2009)
• Homework 4 - PDF (due April 23, 2009)
• Final Project - PDF (due May 13, 2009)

Resources:

• Perl resources
• Perl Tutorial
• IND (machine Learning package)
• Harvest User Manual (HTML mirror)
• Harvest User Manual (PS + PDF via Citeseer)
• HTTP - WWW protocols

Topics:

• Information Retrieval - Topics include a comprehensive study of current document retrieval models, mail/news routing and filtering, document clustering, automatic indexing, query expansion, relevance feedback, user modelling, information visualization and usage pattern analysis.
• Text Understanding - This segment of the course will focus on additional language processing steps for template filling and information extraction from retrieved documents, including reference resolution, sense tagging and summarization. Emphasis will be placed on recent, primarily statistical methods.
• Web Agents and WWW Applications - The final segment of the course will explore current issues in information retrieval and data mining on the World Wide Web. It will focus on case studies of web agents, spiders, robots and search engines, exploring both their practical implementation and the economic and legal issues surrounding their use. One of the hot technologies of the 21st century!