CS 120 / Lab 7 - April 2010 Allison Mankin, mankin@cs.jhu.edu Today we'll discuss how we test programs. These lab notes contain excerpts of an excellent article about unit testing, functional testing, distinctions between them. It also has some tips how to write tests. Don't take Canna's article as dogma. Notably, it represents a practice called "Extreme Coding" and this fits some projects but not all, and not necessarily classroom projects. On the other hand, some of the class already uses ideas that Canna recommends! After we discuss the Canna article, we'll go round robin through the class with everyone (really everyone) talking about their experiences and ideas for testing. Share with classmates testing methods that have worked well or maybe not so much...Are there problems that classmates might help with? Links: Jeff Canna, Testing, fun? really? http://www.ibm.com/developerworks/library/j-test.html Google's open source for automated C++ testing --> Start on this page: http://googletesting.blogspot.com /2008/07/announcing-new-google-c-testing.html Article Excerpts ---------------- [Excerpted quotations are from cited webpage and are purposed 100% for classroom instruction.] Jeff Canna, Testing, fun? Really? Using unit and functional testing in the development process, 2001-03-01 Unit tests should become central to how you write code, especially if the project you are working on has tight time constraints and you'd like to keep it under control. Unit tests are so important that you should write your tests before you write the code. A maintained suite of unit tests: * Represents the most practical design possible * Provides the best form of documentation for classes * Determines when a class is "done" * Gives a developer confidence in the code * Is a basis for refactoring quickly Unit tests constitute design documentation that evolves naturally with a system. Read that again. This is the Holy Grail of software development, documentation that evolves naturally with a system. What better way to document a class than to provide a coded set of use cases. That's what these unit tests are: a set of coded use cases that document what a class does, given a controlled set of inputs. As such, this design document is always up-to-date because the unit tests always have to pass. You should write tests before you write code. Doing so provides a design for the class that the test will exercise, allowing you to focus on small chunks of code. This practice also keeps the design simple. You aren't trying to look into the future, implementing unnecessary functionality. Writing tests first additionally lets you know when the class is complete. When all the tests pass, the task is complete. Lastly, unit tests provide you with a high degree of confidence, which translates into developer satisfaction. If you run unit tests whenever you make changes to code, you'll find out immediately if your changes broke something. Functional tests are even more important than unit tests because they verify that your system is ready for release. The functional tests define your working system in a useful manner. A maintained suite of functional tests: * Captures user requirements in a useful way * Gives the team (users and developers) confidence that the system meets those requirements .... Unit versus functional tests Unit tests tell a developer that the code is doing things right; functional tests tell a developer that the code is doing the right things. Unit tests Unit tests are written from a programmer's perspective. They ensure that a particular method of a class successfully performs a set of specific tasks. Each test confirms that a method produces the expected output when given a known input. Writing a suite of maintainable, automated unit tests without a testing framework is virtually impossible. Before you begin, choose a framework that your team agrees upon. Functional tests Functional tests are written from a user's perspective. These tests confirm that the system does what users are expecting it to. ... As mentioned earlier, you should write unit tests before you write the code they will test. How can you write a test for something that doesn't exist yet? Very good question, Grasshopper. Mastering this practice is ninety percent mental and ten percent technical. What I mean is that you simply pretend that the class you are writing the test for exists. Then write the test. Initially you will get a lot of syntax errors, but stay with it. What you are doing through this exercise is defining the interface that the class will implement. The next step is to run your unit tests, fix the syntax errors (that is, implement the class with the interfaces just defined by your test), and run the tests again. Repeat this process, each time writing just enough code to fix the failures. Run the tests until they pass. The code is "done" when all of the unit tests pass. In general, there should be a unit test for every public method of your class. However, methods with very straightforward functionality, for example, getter and setter methods, don't need unit tests unless they do their getting and setting in some "interesting" way. A good guideline to follow is to write a unit test whenever you feel the need to comment some behavior in the code. If you're like many programmers who aren't fond of commenting code, unit tests are a way of documenting your code behavior. Put the unit tests in the same package as the associated classes being tested. This type of organization allows each unit test to call methods and reference variables that have access modifiers of package or protected in the class being tested. ...