Demjava regression tests. Quick Start: To run regression tests on a version of demjava set up your demeter variables (DEMJAVA_HOME, etc) to point to the version of demjava that you'd like to test and type 'RunRegression' in this directory. Usage: The RunRegression script takes one argument, the path to the directory that contains the test suites it should run. It runs all the suites in its target directory, prints a summary of their results and exits. A test suite is a directory whose name begins with the prefix 'suite-' which contains any number of tests. A suite is executed by running each of the tests it contains. A suite is successful iff all of its tests execute sucessfully. A test is a directory whose name begins with the prefix 'j-' which contains a make file with the targets 'make clean' and 'make test'. A test is executed by running 'make clean' followed by 'make test' in its directory. A test is successful if the word 'SUCCESS' and none of the following list of words appear in the output of 'make test': Parse error Syntax error Compilation Failed Invalid NullPointerException Memory fault FAILURE The argument to RunRegression can be a directory that doesn't contain any suites but does contain some tests. In that case, the tests are run. Each test's 'make test' output is saved in a file named 'output' in its directory. (if the test is named j-bypassing1, its output will be left in the file j-bypassing1/output). Adding New Regression Tests: Make a directory with a name that begins with the prefix 'j-'. Put a makefile in it with the targets 'clean' and 'test'. Make the 'test' target print 'SUCCESS' if the test succeds and 'FAILURE' if it doesn't. Finally, move the test directory into a suite directory if you feel like it. NEW--- Now 'RunRegression' does what it used to previously, and in the end shows all the failures(only). It also creates a file "results.failure" In order to see only the new failures, create a file "known.failure" and run 'CheckFailure' . "known.failure" is a simple file containing the path of each test-case known to fail. A simple way to create the log "known.failure" is to run the script 'MakeKnownFail', which creates the file from "results.failure". If required, you can later edit the "known.failure".