MLM Directory

Do all successful safety-critical computerization projects have automated testing ?

<<Level 1: Accidental Automation. The first level of the software testing model--like level 1 in the software process model-- is characterized by ad hoc, individualistic, chaotic attempts to get the job done. Important information (for example, what to test) is not documented and must be extracted from in-house experts. Test plans are sketchy. Test results are not documented consistently. Schedules slip. Either products are delayed or testing becomes a cursory, poorly documented exercise. Management is uninvolved or uninformed. This level has been designated Accidental Automation because the use of any automated tools or techniques comes about almost as if by accident and is not supported by process, planning, or management functions. Products released on the basis of such testing may well be accidents waiting to happen. Testing at this level may be appropriate only for a product that has no potential for harming the patient or user; it is never appropriate for a computerized medical device. Level 2: Beginning Automation. The second testing level corresponds directly to Level 2­Repeatable in the software process maturity model (see Figure 2). There are hundreds of capture-and-replay test tools on the market today that simply repeat the responses of a system under test.5 As in the process model, however, these tools have limited capabilities and lose their economic usefulness quickly as a product changes. Level 2 testing is still dependent on information locked in the minds of in-house experts, although documentation is beginning to appear in the form of software requirements specifications (SRSs) and test requirements specifications (TRSs). However, in most cases, large portions of these documents are written after the fact and used to meet regulatory requirements rather than to direct the development and test processes. Writing them does, however, provide good practice for moving to level 3. Level 3: Intentional Automation. At the third level, automated testing becomes both well defined and well managed. The TRSs and the test scripts themselves proceed logically from the SRSs and design documents. Furthermore, because the test team is now part of the development process, these documents are written before the product is delivered for testing. Consequently, schedules become more reliable. Level 3 is appropriate for many medical device manufacturers. Level 4: Advanced Automation. The highest testing maturity level is a practiced and perfected version of level 3 with one major addition: postrelease defect tracking. Defects are trapped and sent directly back through the fix, test creation, and regression test processes. The software test team is now an integral part of product development, and testers and developers work together to build a product that will meet test requirements. Any software bugs that do occur are caught early, when they are much less expensive to fix. When testing is performed at this level, an FDA inspector can pick up any piece of product documentation and trace the development process all the way from the SRS that describes the feature to the test results that validate it.>>

Public Comments

1. My dear Shariput.... Well...to be honest...I am a total "Zero" when it comes to the technology of a computer (and all technology in general)-smile- but I assume...they would/should have "automatic testing" features...but...I shall ask my neighbor (who's a little "wizz" with the computer) and try to get back to you later.... Send you all my love & care...your friend....Annette***