Testgen is a computer program that generates suites of input and configuration vectors for testing other software or software/hardware systems. As systems become ever more complex, often, there is not enough time to test systems against all possible combinations of inputs and configurations, so test engineers need to be selective in formulating test plans. Testgen helps to satisfy this need: In response to a test-suite-requirement-specification model, it generates a minimal set of test vectors that satisfies all the requirements.
Testgen generates test cases following a combinatorial approach, but instead of generating all possible combinations across all test factors, it generates a test suite covering all possible combinations among user-specified groups of test factors. Testgen affords three main benefits:
- The level of coverage of the test space can be increased or decreased easily by modifying the test model. Hence, the rigor of testing can be adjusted according to availability of time and resources.
- Within a test model, degrees of combinations can be adjusted separately for different subsystems.
- Typically, Testgen generates test cases in seconds, whereas manual generation of the same test cases takes hours, and Testgen never omits desired combinations or includes redundant test cases.
This program was written by Anthony C. Barrett and Daniel L. Dvorak of Caltech for NASA’s Jet Propulsion Laboratory.
This software is available for commercial licensing. Please contact Karina Edmonds of the California Institute of Technology at (626) 395-2322. Refer to NPO-45921.
This Brief includes a Technical Support Package (TSP).
Combinatorial Generation of Test Suites
(reference NPO-45921) is currently available for download from the TSP library.
Don't have an account?
Overview
The document is a Technical Support Package from NASA's Jet Propulsion Laboratory, focusing on the combinatorial generation of test suites, specifically under the reference NPO-45921. It addresses the challenges of testing complex systems, which often involve numerous combinations of options, inputs, and configurations. Given the increasing complexity of systems, traditional exhaustive testing is often impractical due to time constraints.
To tackle this issue, the document introduces a tool called "testgen," which generates a reduced or targeted set of tests. This tool is designed to cover all combinations of interacting test factors specified in an input test model, rather than attempting to cover every possible combination across all test factors. The generated test suite is structured to meet four key properties: it includes all user-specified test cases, excludes any user-defined cases, covers all specified n-way combinations that are not excluded, and avoids redundancy unless explicitly stated in the model.
The document outlines the syntax of the test model, which consists of various line types, including comments, titles, test factors, and directives for including or excluding specific test cases. Each test factor must be defined with its levels, and the order of these factors is crucial for the proper functioning of include and exclude directives. The document emphasizes the importance of understanding the relationships between factors, particularly when the existence of one factor depends on the value of another.
Additionally, the document provides an example of a test model that illustrates the structure and components necessary for effective test suite generation. It highlights the significance of combinatorial testing, which allows for efficient coverage of potential interactions among factors without the need for exhaustive testing.
Overall, this Technical Support Package serves as a comprehensive guide for utilizing combinatorial generation techniques in test suite development, offering insights into best practices and methodologies that can be applied in various technological and scientific fields beyond aerospace. It aims to facilitate the effective testing of complex systems, ultimately contributing to advancements in technology and innovation.
