Bill Jackson

Searching for defects amid several thousand lines of code in mission critical software, NASA’s Independent Verification and Validating Facility (IV&V) was open for business in 1994 as a safeguard against mission failure. Reporting to the Goddard Space Flight Center, the IV&V audits software across NASA (and other government agencies) dealing with several different projects concerning satellites and shuttle mission software. The current Deputy Director, Mr. Jackson was Acting Director of IV&V from January to October of 2006.

NASA Tech Briefs:

As Acting Director, how did you approach your role at IV&V?

Bill Jackson: The director of the facility has a number of roles. First of which is the facility itself — the director is responsible for the care and feeding of about 44 civil servants and a building NASA built back in the early 90s. Organizationally, the types of things that we report back go to Goddard as part of the Goddard organization.

Second is the role of IV&V Program Manager. He or she is responsible for the delivery and assessment of IV&V services to NASA. And, basically, what that means is, for mission or safety-critical software as defined by the agency, we are an additional layer of assurance that goes on that says that when we are through, the software we looked at — and we don’t look at all the software — will meet the mission needs, and we don’t see any problems with the software impacting the primary or secondary mission objectives.

NTB: What roles does IV&V play in NASA’s structure?

{ntbad} Jackson: We have two objectives. One is the assurance statement that I alluded to, which states that at the final review held by the chief engineer and the chief safety mission assurance officer, we have to stand up and identify what software we looked at and any issues or concerns we have with that software. We make a statement that says we believe that what we looked at will meet the needs of the mission as the project and the agency define it.

We also try to get involved with projects as they move along their software development lifecycle, and identify early any problems we have to the project itself, so that as issues are introduced, they can be identified by us and resolved in an effective manner by the project. What happens is, typically projects will have issues found when they are in the test phase, and could relate to a requirements issue. We try and prevent that, because the cost to fix something when you find it late in the lifecycle is orders of magnitude more than if you detect it in-phase with the development lifecycle.

NTB: How does the IV&V validate and verify a project?

Jackson: We have a standard process that we tailor to the characteristics of a given mission. We start with what we call a criticality assessment analysis, where we look at everything in the software world, and try to determine whether it is critical and/or risky to the program and we concentrate our efforts in that. We never look at all the software associated with the mission, just the things that come out of our criticality assessment as being where we ought to put the focus. Then we start off in the earliest times looking for requirements as those requirements are being formulated. We try to find requirements issues and get them identified and resolved before software moves out of the requirement stage. Then we follow the development lifecycle, looking at development artifacts. We’ll look at, after requirements, design artifacts. Then we will actually get the code in, doing our analysis on the code using manual inspection techniques as well as with some analysis tools that we’ve built. We will look at the testing program. So if there is a three-year development lifecycle for the software in a given project, we’re involved with that project for the full three years.

The final steps are that usually the projects have their Milestone Reviews. The last one is the Mission Readiness Review, where the project managers ask us to come back and report to them on whether we think there are any issues with the software or not. Then, at the agency level, there is a final review with the chief engineer and the chief safety mission officer where we do the same thing. We have a number of techniques we have developed over the years, all of which are traceable to an IEEE specification, the IEEE Standard 1012 for Software Verification and Validation. We have a number of tools we’ve built that we apply, a number of commercial tools, that we use to help us with the job.

We have some really smart and inquisitive analysts, but it is not a team of engineers at their computers going over individual lines of code. In fact, by the time we start looking at actual code, we understand the requirements. We think the requirements are right, all the way from the system-level requirements that get allocated to software; we look at that allocation process. We understand how the code was intended through the design process. Then we look at the code, and how that code is being tested to verify that it will meet the mission needs. One of the things we look for from a testing standpoint is to try to identify areas that the project workers may not look at, and so demonstrate where we think that there might be some weaknesses.

NTB: With what sort of projects is IV&V involved?

Jackson: We supported 23 missions so far. They ranged from the space station through the rest of the human-rated software endeavors, both mission and shuttle. The shuttle is flown by software. It is automated until it gets down to about 10,000 ft. in re-entry. Only then do the pilots take over for the actual touchdown. We are about to get involved in Constellation, which will be another human-rated project. We look at the various robotic missions that are sponsored and being developed under the science mission directorate. We’re doing some analysis on the IEMP, the Integrated Enterprise Management Program. It’s the big financial system that NASA is putting in place that is SAP-related. It’s one the ERP (Enterprise Resource Planning) projects.

We have been involved with missions being developed or software being developed at every center at NASA with the exception of Stennis. We have not been involved with any projects at Stennis, but we’ve hit the other 10 centers. Our focuses are on the JPL, Goddard, and JFC/KSC.

NTB: How could IV&V technology and protocols be applied commercially?

Jackson: The only change to what we would do is the definition of what the mission-critical aspects of the software would be. If you have a financial system, it has different end-goals that define what makes it critical than those of human space flight. But all our processes would be identical to what on goes on in the commercial world.

That’s one of the reasons we’ve had dialogue — more an information-share — with other government agencies, like the Department of Homeland Security, the Department of Defense, the National Security Agency, and our counterparts with the Japan Aerospace Exploration Agency and the European Space Agency.

NTB: What are the goals of IV&V?

Jackson: Our number-one objective is to satisfy NASA’s needs. What that means is, we deliver high-quality IV&V services, analysis, and early identification of errors, making sure we have the right coverage so we can assure that the software will work. We want to be a pre-eminent leader in the discipline of software IV&V, and that’s the reason we look for opportunities to dialogue and collaborate with other agencies, both international and US government agencies in activities that will help us both learn.

One of our basic goals is to be recognized as a leader in the world of independent verification and validation, and so we’re always looking to share our knowledge and techniques with the rest of the people that are developing software that is critical to their mission ends.

We have a research component that is looking to advance the state of the art in how we do independent verification and validation of software. We have initiatives that cut across industry, all of NASA, and the academic world to help us identify the needs new methodologies, and tools to help sell some of the existing processes, and what may be down the road, in terms of how NASA builds missions. One of the things we did here over the last three years is that NASA has started to get into artificial intelligence-type expert system neural networks, and those end up with non-deterministic solutions coming out of the software, which is something brand new for us. So we invested about three years in a research initiative to get us started, so that five or 10 years down the road, if NASA moves over to more employing of non-deterministic projects, we’ll have a pretty good idea how do to the IV&V on those new technologies.

The last goal, and I don’t mean to trivialize this, is outreach. We feel an obligation to the community, especially in West Virginia, to stimulate students to want to pursue technical careers. We have an Educator Resource Center to train teachers in how to teach students. As part of that outreach, we are planning to have 900 7th-graders from all over northern West Virginia here to get some hands-on experiences with NASA. We’re going to have an astronaut talking to them; we do this once a year as part of IV&V’s outreach activities.

NTB: Goddard oversees IV&V; how do the two entities integrate?

Jackson: Goddard is the organization. We are Code 180 in Goddard. Goddard provides us with institutional resources, like where we get our HR activities, procurement activities, and the kinds of things that enable us to go do the work. We report out in a programmatic sense to Goddard on a monthly basis, like the rest of the programs at Goddard. The functional management of the IV&V program is actually vested in the Office of Safety and Mission Assurance at NASA Headquarters. They allocated it to Goddard; Goddard assigned this organization, the NASA IV&V Facility. Goddard does some overseeing on a monthly basis.

NTB: How has IV&V met NASA’s reorganization?

Jackson: That really didn’t affect us much at all. We were already functioning at agency-level, across all mission directorates. The challenge we have, which is the same challenge the rest of NASA has, is from about fiscal year ’08 through fiscal year ’10, when we are trying to build the space station, continuing to fly the shuttle, and start to build the Constellation pieces. There is a lot of demand for resources during that period. That’s one of the challenges we have now, is to figure out for that three-year period how we’re going to deal with the resources that NASA has allotted to us. We’ll have a little different philosophy in how we do things over that three-year period until the shuttle program ends in 2010.

And then it will be Constellation. This year, we started working. We have some civil servants that are working now. We’ll probably bring in some contractors in the Spring, and start gearing up to support the Constellation mission. The official date for Constellation’s launch is 2014, even though they are trying to push that back in.

For more information, contact Bill Jackson at This email address is being protected from spambots. You need JavaScript enabled to view it. .