Regression Test Harness and Hudson Continuous Integration System
Hudson the continuous integration system, has an inherent capability of displaying trends and other historical information of unit tests, provided the xml conforms to the ant's junit task output schema.
jtreg with the option -xml does just that, and thus
makes it easy for the Hudson job to generate and render reports of
Here is how with 2 simple steps
Step 1: In your Hudson job configuration, add a shell or ant script
An example shell script
( java -jar jtreg/lib/jtreg.jar \ -automatic \ -xml:verify \ -agentvm -concurrency:8 \ -v:err,fail \ -r:build/jtreg \ -w:build/jtreg/work \ -jdk $SDK \ test/tools/launcher \ closed/java/awt/SplashScreen \ test/tools/pack200 || exit 0 )Note:
-xml:verify will have
xml reports, as well as verify the xml conformance of the output
files. It is far easier to debug any errors now than later when the
same validation is done by Hudson.
jtreg upon encountering a failure will return a
non-zero exit value, this will terminate the job, and will prevent
data collection by Hudson. Therefore the test is run within a
sub-shell guaranteeing a successful exit value. Not to worry, test
failures will cause the job to be flagged as "unstable".
Step 2: Data collection
As shown below add the entry
build/jtreg/work/**/*.jtr.xml, in the Hudson Job
configurations page, this would be the
directory specified with the
-w option in the above
Thats it!, after a few builds you should be able to see similar results as shown below, this makes it easy to scan for test failures and drill down to get more information, such as test duration, historical failures, test output etc.
Click on the images below to enlarge