When to stop development testing: An infrequently used stopping rule revisited

During development testing and test-analyze-fix programs, the question often asked is when is it suitable to stop testing. Although there have been a number of stopping rules proposed for software development, few have been developed for hardware. Apart from MIL-HDBK-781A and extrapolation by way of a reliability growth model, few other rules exist for electronic systems comprising hardware and embedded software. There are, however, a number of limitations to both these traditional approaches and many of their assumptions are questionable. This article looks at one of the software stopping rules. identifies how it works for electronic systems, and proposes a modification which makes it more suitable. Simulation results are presented for the exponential, Weibull, and nonhomogenous Poisson process models which show that its application in typical system environments is comparable and possibly superior to the traditional methods. In addition, as testing can commence much earlier in the development cycle, th e stopping rule results in less impact to the product's critical path. From a practical viewpoint, the results of application of the rule are presented for two products.