600.465 Syllabus: Introduction to NLP (Fall 1999)

Instructor: Jan Hajic	TA: Gideon S. Mann
Email: hajic@cs.jhu.edu	Email: gsm@loon.cs.jhu.edu
WWW: http://www.cs.jhu.edu/~hajic	WWW: http://www.cs.jhu.edu/~gsm
Office: New Engineering Building 326	Office: Graduate Students Office, NEB 332
Phone: (410) 516-8438	Phone: (410) 516-4650
Office hours: Mon 10-11, Tue 3-4 and by appointment	Office hours: Thu 2-3 and by appointment

New on this course's web pages: (exception: regular class slides posted without warning, usually the night before the class)

• 1/6/00  Make-up exam for "Incomplete"s will be given on Wednesday, Jan. 19, at 2pm. Please be ready and come to my office at 1:55pm.
• 12/14/99  "Big example" for shift-reduce parsing added to the slides for the class of 11/22.
• 12/06/99  "Small example" for shift-reduce parsing added to the slides for the class of 11/22. Please look at it carefully - something similar (only a tiny bit more difficult) might be part of the final exam.
• 12/05/99  Missing slides uploaded. Everything should be current now.
• 11/10/99  Deadline for Homework #3 extended until Dec 8. You're supposed to turn in a perfect homework, though! :-) Homework #4 will consists solely of noun phrase chunking under Grace Ngai's supervision; there will be a report to turn in, too (by Dec 13 as advertised), even though there might be separate deadlines for the parts of the experiment(s). Exact rules/guidelines will be distrubuted next week (11/17).
• 11/10/99  Yesterday's slides split; new slides & changes for today.
• 11/03/99  Hints for installing Eric Brill's tagger on hops added to the Assignment #3, Part 1 description.
• 11/02/99  Corrected foils for classes 4 (removed PowerPoint's confusing dash in front of Kullback-Leibler distance definition), 12 (error in update formula for s_k-1), 13 (HMM initial state added to S and consistently changed elsewhere into lowercase s₀) and 18 (wrong description for the output Y set (tags->words)) uploaded.
• 11/01/99  Midterm exam posted (Questionnaire, Answer Key).
• 10/29/99  New Homework #3 Czech data (hops:~hajic/cs465/textcz2.ptg; extra tags found in the originally posted data). Of course, if you have not downloaded the data yet (i.e. before 7:20pm 10/29), ignore this.
• 10/28/99  Base mutual information for part 2 of homework #2 corrected again; now it's really final (more independently written programs agree on the values.)
• 10/28/99  Eric Brill's tagger location specified explicitly (some links from his home page do not work.)
• 10/27/99  Grading weights changed (slightly in favor of the homeworks.)
• 10/27/99  Homework #3 description posted.
• 10/27/99  Changed class 9 (9/29) description (deleted broken link)
• 10/27/99  Uploaded new version of Classes (programming tricks and tips), class 12 (10/6). Contains corrected formula for L_k-1(i,j) (from Brown's paper) & more.
• 10/26/99  Base mutual information for part 2 of homework #2 corrected.
• 10/25/99  Changes in lateness policy have been posted.
• 10/22/99  First five merges and a base mutual information for part 2 of homework #2 posted to make the debugging process easier.

Prerequisites & Relation to Other Courses:

The material covered in this course is selected in such a way that at its completion you should be able to understand papers in the field of Natural Language Processing, and it should also make your life easier when taking 600.466, 600.666 and eventually 520.779 (although it is not a prerequisite for them).

No background in NLP is necessary.

Readings:

• Required:

  • Manning, C. D. and H. Schütze: Foundations of Statistical Natural Language Processing. The MIT Press. 1999. ISBN 0-262-13360-1.

  • Allen, J.: Natural Language Understanding. The Benajmins/Cummings Publishing Company Inc. 1994. ISBN 0-8053-0334-0.

• Recommended & Reference Readings:

  • Wall, L., Christiansen, T. and R. L. Schwartz: Programming PERL. O'Reilly. 1996. ISBN 1-56592-149-6.

  • Cover, T. M. and J. A. Thomas: Elements of Information Theory. Wiley. 1991. ISBN 0-471-06259-6.

  • Charniak, E.: Statistical Language Learning. The MIT Press. 1996. ISBN 0-262-53141-0.

  • Jelinek, F.: Statistical Methods for Speech Recognition. The MIT Press. 1998. ISBN 0-262-10066-5.

  • Proceedings of major conferences (related to Natural Language Processing):
    • ACL (Association of Computational Linguistics)
    • European Chapter of the ACL
    • COLING (International Committee of Computational Linguistics)
    • ANLP (Applied Natural Language Processing, by ACL)
    • ACL SIGDAT, SIGNLL other SIG (Special Interest Groups) Workhops, such as WVLC (Workshop on Very Large Corpora)
    • EMNLP (Empirical Methods in Natural Language Processing
    • DARPA HLT (Defense Advanced Research Project Agency Human Language Technology Workshops)

• Local Resources:

  • CLSP Workshops: Language Engineering for Students and Professionals Integrating Research and Education

Assignments & Due Dates:

Turning in the Assignments

• When

  The assignments are due at the beginning of the class on the due date, i.e. at 2pm.

• How (technically)

  • Use a separate directory for each assigment. Create a main web page called index.html or index.htm in that directory. Create as many other web pages as necessary. Put all the other necessary files (.ps files, pictures, source code, ...) into the same directory and make relative links to them from your main or other linked web pages. If you use some "content creation" tools related to MSFT software please make sure the references use the correct case.

  • Pack everything into a single .tgz file:

    cd <your assignment #x directory>
    tar -czvf ~/assignx.tgz ./*

  • Send the resulting file by e-mail to

    cs465@peregrine.cs.jhu.edu

    with the following subject line:

    Subject: <you SSN with hyphens> <number of the assignment>

    e.g.

    Subject: 123-45-6789 2

    for somebody having the SSN 123-45-6789, turning in the second assignment.

Policies

• Lateness

  You have up to 10 (ten) late days to use up, which can be distributed among your homeworks. However, the last homework (#4) must be turned in 2 days before the final exam at the latest (2pm, 12/19), even if you have not used all of your days by then.

  Once you use up your late days, late homeworks will not earn any points, even though they might be considered in borderline cases for the final grade. Thus try to turn in all homeworks, even though you might feel they are not to be counted.

• Plagiarism

  No plagiarism will be tolerated. If the assignment is to be worked on on your own, please respect it. If the instructor determines that there are substantial similarities exceeding the likelihood of such an event, he will call the two (or more) students to explain them and possibly to take an immediate test (or assignment, at the discretion of the instructor, not to exceed four hours of work) to determine the student's abilities related to the offending work. *All* cases of confirmed plagiarism will be reported to the Student Ethics Board.

The Assignments

Additional resources:

• Eric Brill's short guide to Perl.
• Extensive material on Perl 5.
• Note on using Perl on the NT machines in the lab.

Tentative Course Schedule:

Course Requirements and Weights:

Exams: