I've been asked to give an impromptu talk about unit-testing to another team. I looked over our test codebase for inspiration as to what points I should cover...
Distinguish between integration testing and unit testing. It is key that units can be isolated. Interfaces are the tool to achieve this. Good unit testing can reduce the need for integration testing.
Unit testing: automated vs. manual, every build vs. infrequently (e.g. load-testing, performance-testing, torture-tests)
Use logging to manually review complex processes (e.g. pooling, threading, resource acquisition and release, synchronization)
Even in mature systems, write unit tests for small pieces as you go in to touch them. Don't be discouraged just because you can't get full coverage.
Instrument user interfaces to record and playback test cases. It can then be much simpler to write test cases.
Create simple (unpublished) interactive interfaces to complex components so that tests can be made up and run on the spot. Such a facility often suggests new automated test cases. And there is nothing like human eyes to spot errors that even carefully design test suites fail to anticipate.
Try to keep tests self-contained. Test database components against local databases (e.g. Access files, In-memory databases, etc). It is frustrating when the nightly build fails because a unit test fails when some externally managed server goes down -- or the data changes.
Use non-deterministic tests to test performance requirements, or to detect memory leaks, dangling pointers, etc. In the latter case, running the test suite itself a few hundred times (minus the leak test, of course) provides an excellent test environment.
If you must use external databases, try to generate the test data with each test run.
Use all of your normal programming ingenuity to write comprehensive test cases. Create supporting test frameworks. Use O-O techniques to capture commonality between test cases. Use data structures to, for example, compare two randomly sorted result sets for equality. Assemble a library of useful test support routines, within projects and across projects. Use good algorithms to generate credible test data and to verify the results. Dispose of temporary resources, especially persistent ones like temporary files or database tables.
Document test suites so that they read like detailed specifications.
Use the same test cases against all implementers of an interface. This may necessitate mandating that implementers must have associated test code that will produce and consume standard data sets. The interface test code should first verify that the implementing component meets this criterion before running the rest of the tests.
Leave hooks in the real code that test code can exploit. Strive to keep the hooks private to the component, especially in components that have security roles.
Use a development environment that allows test cases to be run piecemeal, preferably avoiding a full compile-link-load-run cycle, and avoids running the whole test suite (or system) just to run one test. A tight cycle reduces the temptation to skip running (or writing) tests.
If your environment does not have a testing framework, write one. They are not too hard, and you can grow it incrementally.
Stay focussed in your testing. Just because a component is usually invoked within a certain environment, does not mean that a test has to take place there. All you have to do is simulate the environment (but make sure your simulation is accurate!). Test cases grow exponentially with component complexity, so keep components small. Design for isolated testability. Good unit tests for the containing environment will test the integration.
Try to write good black box tests, that have complete coverage, but do not neglect white box tests. Some possible test scenarios are not identifiable from outside a component, such as strategy changes.
Make tests repeatable. If generating random data, make sure that the seed is either hard-coded or at least reported so that a failure can be reproduced as necessary. If creating simulated load or concurrency conditions, use positive measures to ensure repeatability (such as explicitly synchronizing the events in two threads instead of leaving their coincidence to chance, or relying on certain performance characteristics [e.g. clock speed] of the test machine).
Avoid enshrining accidental features of an interface in the test code. For example, if an interface returns a list of items with no ordering guarantee, don't rely on the fact that the current implementation happens to return them in a certain order.
Use UI robot technologies to test UIs. Don't be afraid to write code that injects simulated events to your UIs -- in many frameworks this is not terribly hard to do.
Verify that your tests fail when erroneous behaviour occurs. If you write your tests before you write the code that implements the tested functionality, you get this check for free -- until the implementation occurs. After that, this situation becomes more difficult to verify. There exist utilities that provide automation for this task, 'breaking' unit tests by changing subtle features in the tests like expected literal strings and so on, but these automated methods only scratch the surface. This is an outstanding unsolved problem. Occasionally break a test on purpose, just to make sure things are acting as expected.
Use instrumented implementers of interfaces with 'dispose()' and 'close()' methods to verify that infrastructures that use them do, in fact, dispose of the objects exactly once.
Test source code is *real* source code. Keep it in the project tree. Keep it in the version control system.
Parameterize variable and sensitive test configuration items, such as database server names, user ids, passwords, etc, and make it possible to configure these items externally when the test is run. (e.g. command line arguments, environment variables, INI files, etc).
There is no need to exhaustively test every capability of a third-party library -- unless you are publishing all of that capability as part of your system. More likely, your system only uses a fraction of the third-party library's functionality. Test that, either directly, or indirectly by means of the test cases that verify the dependent component's behaviour. Better still, wrap the third-party library in a new facade component and test that. It provides a more controllable dependency, and may even permit replacement of the third-party library by one from a different vendor should conditions demand it. The same reasoning holds for other third-party items such as SQL statements, database drivers, and so on.
Don't worry about testing functionality in any given order, e.g. from least complex to most complex, or in reverse dependency order -- just test!
