Safety-critical development standards, such as DO-1781, have been very successful in guiding the production of reliable software. These standards assist developers in applying good software practices during development, ensuring, as much as possible, the absence of errors. Compliance with these stringent standards involves a high cost because of their labor-intensiveness and high levels of know-how.
One aspect that has been largely neglected in safety-critical development is evolvability. Evolvable systems demand the use of agile engineering processes to hit hard deadlines in shrinking time windows.
Another problem is that system integration comes late and costly. Complex and interconnected systems developed by different groups or subcontractors must integrate successfully, and they need to be readied or adapted quickly to meet changing needs in shorter timescales without forgetting their safety-critical constraints.
A new open-source collaborative initiative, called Open-DO2, tries to increase the agility and the level of automation in the production and certification of safety-critical software, focusing on more continuous and incremental certification approaches. The goal is to provide the processes and infrastructure to guide safety-critical software development in a way that the application and its related material are ready and verified at any time.
The Certification Process
Safety-critical standards focus on the adequate definition and execution of software engineering processes and activities. They define the objectives for the different processes, the activities that need to be performed for achieving these objectives, and the evidences (artifacts) that indicate that the objectives have been satisfied.
Main software development processes which take place in any high-integrity software project are:
- Verification and validation
- Configuration management
- Quality assurance
The planning process determines what will be done to produce safe software conforming to the system requirements. The software development process (requirements, design, production/coding, and integration) is the one that creates the software product. There are three other processes which are orthogonal (integral) to software development:
- verification and validation at the different levels (reviews, analyses, and tests),
- configuration management to identify, control and regenerate the different artifacts, and for appropriate problem tracking, and
- quality assurance to ensure the quality of the software by assessing the software lifecycle processes and their outputs.
The transition between processes is activated when the defined objectives have been satisfied. It requires inspecting these objectives (there will be both manual inspections and automatic verifications), and keeping track of these events and produced materials.
In addition to this, traceability between the different elements is required, both within a process (for example, correspondence between high- and low-level requirements) and among different processes (for example, between requirements and source code implementing the requirements).
The rationale behind certification standards is that the use of standard processes and compliance with predetermined objectives help avoid the common pitfalls of software development. The DO-178 way of ensuring quality is by imposing a level of rigor in the processes and workflows used to build the final system.
From the defined workflow one can infer a temporal and causal dependency between processes, activities and artifacts. For example, editing a requirement shall always precede the verification of that requirement, and the production of the document containing the list of requirements shall always follow the editing and verification of requirements. It is thus possible to infer a set of rules which can be used to check automatically that the workflow has been followed and provide evidence of the engineering process sanity.
Addressing the workflow is one of the objectives of Open-DO. This is one point where Open-DO approaches Lean (focus on flow optimization, elimination of waste and creation of value for the end customer) and Agile (iterative test-driven development) methodologies. Although these two approaches are not widely considered suitable for high-integrity systems, these concepts are actually being successfully applied for developing systems following the DO-178 standard. In a verification-driven lifecycle, where requirements-based testing is a must, the paradigms of test-first and continuous integration allow for appropriate refinement of requirements (from high-level to low-level requirements) and early detection of errors.
The Big Freeze
The essence of safety-critical software development is that all activities and their by-products come with evidence that proper due diligence has been undertaken in the production and verification of such software. As a result safety-critical software comes with a large body of artifacts which can be an order of magnitude more voluminous than the software product itself. When a safety-critical application and accompanying evidence is complete, evolutions to the software often become costly because they entail regenerating the entire evidence-set. As a result, when a piece of software receives the “certified” stamp, subsequent changes are avoided. This is what we call the “big freeze.”
By leveraging on Lean approaches and Agile methodologies we aim, within the Open-DO initiative, to shift the focus of safety-critical software development to more continuous and incremental certification approaches.
Although DO-178 and other safety-critical standards do not recommend specific software development methodologies, software development processes derived from the waterfall model or the V-model are usually employed, and specific activities, such as high-level requirements, low-level requirements, and coding, are carried out in their logical order. Following, or in parallel with, software development, certification activities like verification, quality assurance, and the writing of achievement reports are carried out.
Often, only very minimal and absolutely necessary changes to the software are allowed in the later stages because these changes can have a costly impact on a number of artifacts needed for certification. The result of this is that software can no longer evolve or, to be more precise, will evolve with difficulty as changes invalidate previous certification activities. This is the big freeze phenomenon at the component level.
At system integration level the big freeze can occur too. Behavioral or interface changes in one software component have to be taken into account in all interacting components and their certification artifacts. This may lead to a big freeze at the system level.
Continuous integration is a set of software engineering best practices that emerged in the Extreme Programming community and aim at reducing integration time and cost. It follows a requirement-driven methodology in which first the requirements are specified, then the test cases for the requirements are written, and finally the actual design and code are developed. The application is automatically built after any change, and the test suite is executed.
This way most defects and integration problems are identified immediately, allowing developers to revert their change or fix them right away. Most importantly, continuous integration creates a product where the quality is always high, meaning there is no need to plan for long-running “stabilization” or “verification” phases in the release process.
Although continuous integration is definitely good for certifiable systems, this is not enough. Evolvable and certifiable systems demand a continuous certification process by enhancing continuous integration. The repository would not only contain the sources of the software, but also the whole set of certification artifacts (plans, requirements, design, test cases, documentation, etc.) The goal is to allow for faster certification by automating most of the activities required for certification, so every change triggers a complete run of these activities, signaling those that need to be performed manually.
Introducing agility in the certification process helps having artifacts up-to-date. The traditional method, following a V-model development process with system integration at the end of the project, and certification activities carried out as a separate endeavor, is simply no longer viable for those aiming to develop evolvable and certifiable systems. The underlying idea is to provide the processes and infrastructure to make sure that the application is ready to be certified at any time.
Production of safety-critical systems is typically expensive and not conducive to changes. Agile techniques can help increase the level of automation in production and certification, increasing adaptability to changing requirements and reducing delivery time and cost. These methods are based on iterative and incremental development, verified by continuous and automated tests. This notion can be extended to all certification artifacts to achieve continuous certification.
Open-DO is an open-source initiative which is being put in place to help safety-critical development2. The goal is to create an open ecosystem where the industrial community, tool providers, and public institutions can find the synergies to increase the productivity and flexibility of high-integrity methodologies.
- http://www.adacore.com/home/products /gnatpro/development_solutions/safety-critical/do-178b-overview/