JDK team uses a broad range of test methodologies to develop
test scenarios to qualify the JDK
Engineering writes unit tests to cover sufficient fundamental scenarios to ensure no regressions are introduced in future.
A unit test is a small test that verifies the new code is doing what its author thinks it should do. Regression tests are closely related to unit tests.
A regression test
is a test written in order to verify that a change in existing code
actually fixes a reported bug. Today's unit test is tomorrow's
Java Compatibility Kit is used to
assure conformance of Java SE implementation to the
Thousands of tests, executed in 20+ different combination
||Functional tests are used to
verify the functionality or behavior of the feature under test.
(positive and negative tests)
Thousands of tests. Mostly automated for server side components. For client components includes manual testing for all different look and feels supported by the operating systems.
Based on available code coverage tools, Analysis at frequent interval done to identify areas to focus for additional coverage. (other factors also contribute in final decision on which areas need additional test coverage. e.g. Code churn for the area, customer importance of the area, escalation trend analysis etc.)
|Reliability, performance, stress testing||
JVM reliability and performance
is tested by tracking, executing and publicly posting of results
from running a broad range of public benchmark
Internal stress test tools, random
data generator tools are used to stress test the JVM
Reliability testing is also done on
|Fault Injection testing||
Verification that system responds gracefully to fault conditions and remains operational during unexpected software failures
||Other than ensuring Java platform
compatibility using TCKs, other client and server side
compatibility testing is done.
Unscripted testing left to the tester to attempt to break a feature implementation to find bugs
Bug festivals are arranged at
important milestones where all teams are invited to come and in a
given a specified duration to focus and try to break the product
with with provided tests/tools.
|Customer beta programs
||Apart from internal testing,
targeted and public beta programs are conducted at beta milestones
to gather feedback and strengthen the areas needed.