600.226: Data Structures

Spring Semester 2006: January 30, 2006 - May 5, 2006

Assignment 8: Hashed Speech

Out on: March 31, 2006
Due by: April 6, 2006 by 5:59 pm for full credit (11:59 pm for 10% off, hard deadline)
Collaboration: Pairs
Grading: Packaging 10%, Style 10%, Performance 30%, Design 10%, Functionality 40%

Overview

The eighth assignment for 600.226: Data Structures deals mostly with maps of one sort or another. There are no "written" problems as such, but you can still rant a lot in your README file.

Note that each pair hands in one assignment! Decide early on who is going to be responsible for submitting the assignment and when. Make sure to include all the relevant information (who is in the pair?) in your README file! Both of you will get the same score for the assignment.

Here are the necessary interfaces and exception classes: maps.tar.gz As usual, you are not allowed to change the code we provide in any way! Warning: This is a new version of the assignment and there may be serious bugs in these interfaces. If you think you found a bug, please email the course staff about it immediately. Thanks!

Problem 1: Simple Maps

Your first task is to write a class SimpleMap<K,V> that implements the Map<K,V> interface we provided above. You are free to use the Java classes java.util.ArrayList<T> and java.util.LinkedList<T>. Of course you can also hack your implementation "from scratch" if you prefer, but that will make things somewhat more tedious...

As usual, please provide a toString() method to return a String representation of the map, and a main() method that performs basic unit testing for your implementation. A new map into which the pairs ("Peter", 35), ("Hans", 70), and ("Toni", 67) were inserted should print as {Peter: 35, Hans: 70, Toni: 67} or something close; the order of pairs is not defined.

Problem 2: Hashed Maps

Your second task is to write a class HashedMap<K,V> that implements the Map<K,V> interface using some kind of hash table. You are once again free to use the Java classes java.util.ArrayList<T> and java.util.LinkedList<T> or to start from scratch; in fact, you could even use your SimpleMap<K,V> from the previous problem if it helps... There are a number of options for your implementation, and you may want to consider them a bit before committing to one or the other; you might even decide to change things after you see how your first prototype performs. Here we go:

You can either use chaining or open addressing to resolve collisions; in the former case each "bucket" holds a list which in turn holds entries; in the latter case you can use linear probing or quadratic probing.
You can either go with a fixed size bucket array, or you can grow the bucket array and rehash the content of the map if the load factor of the hash table becomes to large.
The size of your bucket array can be determined in a variety of ways: you can leave it up to your client completely (i.e. just use a size "estimate" passed in the constructor), you can make it a power of two in the ballpark of what your client wants, or you can make it a prime in that ballpark; if you go for open addressing, I strongly recommend the latter.
You can perform the "compression" step using either the division method or the multiply add divide method (with or without randomly determined constants).

Once again, the exact choices are up to you, so you can determine the amount of work you do pretty flexibly. However, please keep in mind that we emphasize performance more than in previous assignments (check the rubric above) so it is in your best interest to make choices that will result in the best possible performance under the widest variety of conditions.

As usual, please provide a toString() method to return a String representation of the map (see Problem 1 for the format). Also, you should again provide a suitable main() method for testing.

Problem 3: Analyzing Speech

A simple way to get an idea of what a certain text is about is to count how often certain words appear in it. Your final task for this assignment is to write a program WordFreq (based on your Map implementations) that performs this kind of analysis.

Your program should accept input text (in plain ASCII format) from standard input and produce a list of the 32 most frequently occurring words on the standard output; for each word, the number of times it appeared should be given as well. Consider the command java WordFreq <in.txt >out.txt. If in.txt contains

Bla bla, balla bla. Balla balla bla
bla bla blue balla bla balla bla.

bla!!

          -- Bla balla blue bla.

then out.txt should contain

bla 11
balla 6
blue 2

and nothing else. As you can see, you should ignore capitalization, punctuation, and white space. In order to get some "meaningful" data out of this, you also have to ignore very frequent words such as "a" or "to" or "or" or "the" or... You get the idea. This site has various lists of "noise words" but I am not sure how good they are; for now I suggest we use their "27 words" as reproduced here:

the, and, a, to, of, in, i, is, that, it, on, you, this,
for, but, with, are, have, be, at, or, as, was, so, if,
out, not

Project Gutenberg is a good source for test data. I suggest Einstein, Kafka, and Marx as simple test cases. Religious texts are more voluminous and thus provide more challenging test cases, for example The Bible or The Koran. Feel free to test on whatever you want, we'll pick our test cases from Project Gutenberg as well.

Deliverables

Please turn in a gzip compressed tarball of your assignment; the filename should be cs226-assign-8-login1-login2.tar.gz with login1 and login2 replaced by your Unix login names on ugradx.cs.jhu.edu. The tarball should contain no derived files whatsoever (i.e. no .class files, no .html files, etc.), but allow building all derived files. Include a README file that briefly explains what your programs do and contains any other notes you want us to check out before grading; don't forget to include your answers to "written" problems as well.

Grading

For reference, here is a short explanation of the grading criteria. Packaging refers to the proper organization of the stuff you hand in, following the guidelines for Deliverables above. Style refers to Java programming style, including things like consistent indentation, appropriate identifiers, useful comments, suitable javadoc documentation, etc. Simple, clean, readable code is what you should be aiming for. Performance refers to how fast your program can produce the required results compared to other submissions. Design refers to proper modularization and the proper choice of algorithms and data structures. Functionality refers to your programs being able to do what they should according to the specification given above; if the specification is ambiguous and you had to make a certain choice, defend that choice in your README file.

If your programs cannot be built you will get no points whatsoever. If your programs cannot be built without warnings using javac -Xlint we will take off 10% (except if you document a very good reason). If your programs fail miserably even once, i.e. terminate with an exception of any kind, we will take off 10%.

Bonus Problem

It is quite interesting to compare the various hash functions people have developed over the years, both in terms of how efficiently you can compute them and how "collision free" they are. For the bonus problem, write a program that compares various hash functions for strings (the most common kind of key out there). Once again you can base this on the SortingAlgorithm framework to some extent: Just define an interface HashFunction instead and feed in text (similar to Problem 3 above); keep track of the hash values you get in a Bag and you can tell how frequently collisions occur; time how long it takes to compute all the hashes and divide either by number of words or number of characters to get a performance measure. This site has lots of hash functions to choose from. You could also try to confirm or disprove the properties claimed for certain hash functions in our text book. As always, we won't give you extra points for this, but we'll give you extra kudos. :-)