Developers run check-in tests on their local build to verify their changes before committing the code to the parent source tree.
Check-in tests must run quickly so that there is no temptation to skip them. They must also be small so that, if the test fails, you can readily determine the cause. In an ideal world, check-in tests achieve 100% code coverage and run in less than 30 seconds.
When they fail, they present you with a test case that is as small as possible and that tests only one thing :-)
For the convenience of those of us (specifically, Herman) who prefer to live inside an Integrated Development Environment and like to pretend that the command line prompt died in the late 1980s, please tie check-in tests to the
xUnit.net test framework
. This allows the tests to be easily run from Visual Studio. The unit test framework makes the cost of running an individual test application high, so it is better to have tens rather than hundreds
of test cases. It is nearly impossible to run thousands of test cases.
CCI Metadata currently includes two test applications, which are located in the source tree under the Tests folder.
- PeReaderTest is an example of how to evade the unit test framework’s limits on large numbers of tests. It uses a specialized test framework that can run large numbers of small test cases very quickly, but that appears to the unit test framework as a single
- RoundtripTest exposes numerous test cases to the unit test framework and already runs uncomfortably slowly. It is probably not scalable enough, so improvements are welcomed.