Avoiding the Pitfalls of Faulty Test Results

No discipline is as precise as the design and manufacture of medical devices. No other function is as crucial in its precision. Exact measurements and tests are a critical necessity— from heart valves to knee and hip replacements, a fraction of a millimeter can be the difference between life and death for a patient. The process is a carefully choreographed sequence of matching design specifications by unit, during which each component is tested, then rejected and scrapped, or accepted. Each step in the process affects all the others that follow, and the process itself affects the quality of life for hundreds of thousands of patients who require a medical device.

Fig. 1 – A technician oversees the quality check process of a medical device during the manufacturing process.
Metrology plays a vital role in the process, measuring not just details, but the precision and correctness of the measurement itself. Metrology enters the process after design and instrumentation stages used to monitor and control the manufacturing process have been selected. The purpose of metrology is to juxtapose instruments with the same specifications with which they were selected. This determines whether or not false results have been reached. Instruments found to be within specifications, a conclusion called “in tolerance”, have a high probability for instrument error that is sufficiently small enough not to induce false results in decisions about the medical device product. If the instrument is found outside of control limits, a process called “out of tolerance” is initiated to investigate whether it caused a defect with the medical device product.

This is critical information. Errors at this stage can trigger false results: a false reject, which leads to a scrap or rework, and is costly to the manufacturer, or a false accept, which can—and has—placed faulty products on the medical device market, the cost of which is significantly higher, up to and including the price of human life. It’s happened. In the 1980s, more than 360 people died due to faulty struts in bileaflet mechanical heart valves. More recently, in 2008, a heart valve device was recalled after discovering a faulty ring that held the valve in place. Thanks to improved quality and testing processes, the defect was discovered in time to prevent human or financial casualties. (See Figure 1)

The variables and processes associated with medical device manufacturing, instrumentation monitoring, and control can introduce a number of errors, any of which, though tiny in actual size, can have serious consequences. Selection of the wrong tool, development of quality blind spots, and operator error are a few of hundreds of potentially disastrous scenarios, causing both false acceptance and false rejection. The danger in false results is that they are generally not evident during the process, which leads to uncontrolled risk, appearing as unwarranted out of tolerance nonconformance report investigations, scrapped product (materials and labor waste), reworked product that unnecessarily expends labor, and the recall of faulty product to avoid consumer injuries or fatalities. These and other faulty results will eventually erode profit margins, an undesirable consequence for all involved. How do these errors occur? What mechanisms can be built in to quality processes to prevent them?

What Can Be Done?

Fig. 2 – Precision instrument selection is crucial for accurate measurement. Shown here: a digital caliper, used to measure inside and outside diameter with accuracy 0.0025 in.
Introduce a two-step process to methodically stem the flood of faulty results: first, identify errors, and second, implement steps to prevent them. To identify errors, pinpoint occurrence spots and identify each error by type. For example, instruments must be calibrated against the specifications for which they were originally considered to be suitable for the intended purpose.

This conclusion may seem unlikely, but consider how errors develop. A manufacturing engineer selects an instrument for a step in the process in accordance with the instrument’s original equipment manufacturer (OEM) specifications. Separately, a calibration coordinator collects all instruments in a facility to calibration suppliers, following company procedures and approved suppliers selected by a separate team of supplier quality engineers. The supplier performs the calibration using their quality system procedures. No other information is shared between the four parties, each believing the others’ outcomes are correct.

The calibration supplier, however, used a military calibration procedure, which deviates from the OEM. Military procedure covers an array of instrument manufacturers and models, such as procedures for digital calipers, pressure gauges, and balances, but their general specifications differ from the OEM. The calibration supplier unknowingly altered the client’s need, and the calibration coordinator may not accurately assess the problem, leaving an in tolerance designation in place.

The manufacturing engineer doesn’t become involved unless there is an out of tolerance result, and likely doesn’t check calibration tolerances against OEM specifications. The quality engineers expect that calibration suppliers maintain quality processes and systems. This common example of calibration disconnect results in a quality blind spot. The mistake? Viewing the process as a singular, forward direction from raw material to finished product. At this stage, risk reduction is early identification of vulnerable parts of the process. Looking backward from the finished product, rather than forward and weighing decisions in the manufacturing process by points of disconnect is made simpler with tools such as 5-Why (See www.isixsigma.com/tools-templates/cause-effect/determine-root-cause-5-whys) and Fishbone Diagrams (www.isixsigma.com/tools-templates/cause-effect/cause-and-effect-aka-fishbone-diagram), which identify error sources that can result in false rejection or false acceptance.


The U.S. Government does not endorse any commercial product, process, or activity identified on this web site.