Software assurance is the planned and systematic set of activities that ensures that software processes and products conform to requirements, standards, and procedures. Examples of such activities are the following: code inspections, unit tests, design reviews, performance analyses, construction of traceability matrices, etc. In practice, software development projects have only limited resources (e.g., schedule, budget, and availability of personnel) to cover the entire development effort, of which assurance is but a part. Projects must therefore select judiciously from among the possible assurance activities. At its heart, this can be viewed as an optimization problem; namely, to determine the allocation of limited resources (time, money, and personnel) to minimize risk or, alternatively, to minimize the resources needed to reduce risk to an acceptable level. The end result of the work reported here is a means to optimize quality-assurance processes used in developing software. This is achieved by combining two prior programs in an innovative manner:
First Program : The first of these programs is the Advanced Risk Reduction Tool (ARRT), which can be used to calculate the costs and benefits of a set of assurance activities on a given software project. ARRT is itself based on a risk-management tool, Defect Detection and Prevention (DDP). DDP uses a detailed mathematical model of requirements, risks, and mitigations.
Second Program : The second of these programs is the TAR2 "treatment learner," which can be used to determine from a large set of factors those factor settings most critical to attaining a given objective.
Innovative Combination : The major contribution of this work is the combination of these two programs. They are combined so as to operate in an iterative procedure, as follows: In each cycle of the iteration, TAR2 is tuned to identify the most critical software assurance activities, both those most critical to perform (because they contribute to cost-effective risk reduction), and those most critical to not perform (because they detract from cost-effective risk reduction).
These identified activities are then set accordingly in ARRT, and the cost-benefit calculations rerun. Repeating this cycle determines more and more activities to perform (and/or to not perform), culminating in a solution that is (near) optimal. An important aspect of this approach is that it allows for human experts to add further guidance during each iteration of the cycle. For example, if the experts observe that two of the recommended activities are actually incompatible (say, because they would both require use of the same limited resource at the same time), they can reject the TAR2 recommendations involving this pair of activities, and instead ask for the next-best solution. This makes good use of the experts' time, since they are only asked for guidance pertinent to promising solutions.
This innovation was developed by Martin Feather and Steven Cornford of Caltech and Tim Menzies of the University of British Columbia for NASA's Jet Propulsion Laboratory
This software is available for commercial licensing. Please contact Don Hart of the California Institute of Technology at (818) 393-3425. Refer to NPO-30512.
This Brief includes a Technical Support Package (TSP).
Software for Optimizing Quality Assurance of Other Software
(reference NPO-30512) is currently available for download from the TSP library.
Don't have an account?
Overview
The document is a technical support package from NASA, focusing on optimizing software quality assurance (SQA) processes. It highlights the challenges faced in software development projects, particularly the limited resources available—such as budget, schedule, and personnel—that necessitate careful selection of assurance activities. The primary goal is to ensure that software processes and products conform to established requirements, standards, and procedures.
The report discusses the importance of cost-benefit analyses in selecting software assurance activities. It emphasizes that projects must quantitatively assess the costs and benefits of various assurance activities to effectively allocate their limited resources. This assessment is crucial for estimating not only budget and schedule but also quality and risk. The document outlines the need for a systematic approach to identify and evaluate trade-offs, such as sacrificing certain functionalities to enhance overall quality.
A key innovation presented in the document is the development of a risk management tool that incorporates cost-benefit calculations. This tool, based on NASA's Defect Detection and Prevention (DDP) framework, allows for a detailed analysis of risks related to software requirements and the effectiveness of risk mitigations. The tool enables project teams to study the implications of multiple interrelated decisions regarding the selection of software assurance activities.
The report also introduces an iterative approach to identifying critical assurance activities. By applying a previously published method for identifying critical values, the approach allows teams to refine their selection of activities continuously. This iterative process helps in finding a near-optimal set of assurance activities tailored to specific project needs.
Overall, the document underscores the necessity of a planned and systematic approach to software assurance, advocating for the use of quantitative methods to optimize resource allocation and minimize risks. It serves as a guide for software development teams aiming to enhance the quality of their products while managing constraints effectively. The work was conducted at the Jet Propulsion Laboratory under NASA's sponsorship, reflecting a commitment to advancing software engineering practices in high-stakes environments.
