600.211: Unix Systems Programming

Fall Semester 2005: September 8, 2005 - December 12, 2005

Assignment 6: Serving the Web Faster

Out on: November 20, 2005 (approximately :-)
Due by: December 12, 2005 by 5:59 pm for full credit (11:59 pm for 10% off, hard deadline)
Collaboration: Teams
Grading: Packaging 10%, Style 10%, Performance 30%, Design 10%, Functionality 40%

Overview

The sixth assignment for 600.211: Unix Systems Programming picks up where the fifth one left off, adding more required features to your very own web server.

This is a team assignment, and all of you should know which team you're in (I hope). Each member of a team will receive the same score for the product you submit together. Note that some parts of this assignment are not 100% about the actual web server code. You'll see... :-)

Problem 1: Profiling your Server

First you need to improve your existing build and test process by integrating the gprof tool to measure the performance of your code. You also need to set up httperf and develop scripts that allow you to test/evaluate your webserver under various synthetic loads. Actually, httperf is probably useful for testing in general, make sure that your gcov setup still works as well. :-) Of course you should integrate both tools into your nightly build process again, and your build logs should reflect the outcome of your performance evaluations as well.

You must submit all your automation/testing/performance scripts and a short description of your setup when you turn in your project!

Problem 2: The Fast'n'Fancy Web Server

Your main task for this assignment is to complete your web server. That might mean different things to different teams, but here is what we are looking for:

Support for all required features of the HTTP/1.0 specification. Please document which parts of HTTP/1.0 you do not support and why; you better have good reasons.
Support for selected features from the HTTP/1.1 specification, specifically persistent connections. Please document which parts of HTTP/1.1 you do support and why. If you decide against persistent connections, give a very good reason for your decision.
Using the profiling and performance evaluation infrastructure you've set up for Problem 1 above, optimize your web server as much as you possibly can (in terms of both speed and memory). Feel free to compare your own web server to the competition. :-) Please write up a short summary of your experience regarding performance tuning: Were you able to find bottlenecks and remove them? Which ones? Why were they there? How did you fix them?
Packaging and documentation consistent with a semi-mature open-source project. If you received feedback about lacking documentation or problems with your tarballs, this is your last chance to fix that. Please spend some time thinking about whether you want to make this into an actual open-source project. I'd highly encourage that: Looks great on resumes, especially if you can actually find users. :-)

Of course, you should also support whatever "special features" you were able to come up with. This is up to you, as it always has been, but we sure hope that there will be something that only your web server can do. :-)

Deliverables

Please turn in a gzip compressed tarball of your assignment (the extension should be .tar.gz). The tarball should uncompress into a directory cs211-assignment-6-teamname with teamname replaced by your team's name (as used for the repository and the mailing list); uncompressing should not create any other files in the current directory. The tarball should contain no derived files whatsoever, but allow building all derived files with make. We expect that your Makefile handles "the usual" targets like clean and test aside from all (which is the main way we will build your program). Include a README file that briefly explains what the program does and contains any other notes you want us to check out before grading; make sure the README lists all members of your team! Include other "common" files such as INSTALL describing how to install your tool, CREDITS to pay your respects to the people whose code you're reusing, and LICENSE to describe copyright and distribution terms if you feel like it. You can look at any number of "famous" open source projects to see what kind of structure is appropriate; gif2png is a relatively small example, but you don't need everything in there. However, seeing as we're now getting close to an actual product, you should give a "real" distribution some thought.

Grading

For reference, here is a short explanation of the grading criteria. Packaging refers to the proper organization of the stuff you hand in, following the guidelines for Deliverables above. Style refers to C programming style, including things like consistent indentation, appropriate identifiers, useful comments, etc. Simple, clean, readable code is what you should be aiming for. Performance refers to how fast your program can produce the required results compared to other submissions. Design refers to proper modularization and the proper choice of algorithms and data structures; often this can be judged by asking "How hard would it be to add feature X?" and "How hard is it to replace algorithm X with algorithm Y?". Functionality refers to your program being able to do what it should according to the specification given above; if the specification is ambiguous and you had to make a certain choice, defend that choice in your README file.

If your program cannot be built you will get no points whatsoever. If your program cannot be built using make we will take off 10%. If your program cannot be build without warnings using gcc -W -Wall -O -pedantic we will take off 10% (except if you document a very good reason). If your program fails miserably even once, e.g. segfaults or runs forever, we will take off 10%.