Test and Verification (T&V) Phase
While T&V can include the hardware platform, that activity is out of scope from the software development itself and, therefore, is beyond the scope of this article. T&V includes regression testing, performance testing, and alpha and beta field trials.
A regression test is the execution of the test cases generated during the implementation phase. An iteration is not considered complete until the associated test case has been added to the regression test suite and the regression test suite is executed successfully. A communication port on the target is used to facilitate the regression testing. A scripted, automated test suite implemented on a port controller (i.e. PC utility) is preferable. The regression test is critical to product quality throughout the software products lifecycle.
The performance of the code should be measured prior to each software release to make sure the latest code development has not negatively impacted the application. At a minimum, the performance measurements should be:
- Memory Usage. A minimum of 10 percent of code and data memory for each memory type should be available at all times to enable maintenance programming.
- Stack usage. Designers should measure the size of worst-case stack allotment. A minimum of 20 percent of stack allocation should be available for maintenance programming.
- Execution time for each interrupt service routine.
- Overhead usage. Developers should determine the worst-case total interrupt execution time as a percentage of a real-time clock interval.
- Buffer Usage. Determine the worst case “high water” mark for each buffer used in the system to make sure 20 percent is available for maintenance programming.
- Timer service overruns.
An alpha field trial is an engineering field trial, not a customer or field support field trial, which includes all elements of the system. It usually is performed prior to software completion, but after the high priority and risk use cases are implemented. Its purpose is to test the product to uncover issues relatively early in the development cycle.
In its best form, alpha field trial is implemented by providing company personnel with product and user instructions. A weekly meeting is held to discuss any questions they have. This provides not only defect detection, but also a usability test of the product and its user documentation.
A beta field trial is a customer (or a field support department) trial conducted in a customer domain. This is performed after all software development is completed. The trial is extended with additional releases if failures should occur. A beta field trial is complete when the customer (or field support department) signs for acceptance of the product.
Maintenance covers the software product support from initial production release to the end of the product lifecycle. The SRD and SAS artifacts from the analysis and design process phases are critical to the cost and quality of maintenance modifications made to the software during this period. They are documents that a maintenance programmer can rely on being accurate and up to date at all times. The same software development phases used in the initial development apply to the maintenance activity.