The term ‘testing’ is something we might have mixed views on. It's something that software developers should have as an integral part of any software project—continually during the project to try to iron out bugs early, and then at the end to ensure deliverable quality. Software rarely degrades after delivery, but strange and unforeseen circumstances can cause errors to pop up—just ask any PC user—so even testing for software failure is rarely complete, even after deployment.

Testing hardware is also something that needs to be carried out periodically post-deployment to check to see if something is about to or already has gone wrong. But when testing spots deviations in service quality and performance small problems can be isolated earlier before they become catastrophic.

In recent years, faults which should have been isolated and dealt with have caused enormous problems: interplanetary space missions have failed due to integration issues with units of measurement between two suppliers; component failures in utility networks have created domino effects that took out entire electricity supply networks; a periodic software upgrade once prevented all of a UK government department's PCs from accessing the network and another simple error appeared to cause a devastating total system failure of French mobile operator, Bouygues. Although not the largest operator in France, the urban coverage and subscriber base meant the communications ability of businesses, individuals and government departments suddenly plunged back twenty years.

The reason we are seeing more unexpected failures, whether as devastating as total network collapse, or the still problematic, but less critical, denial of service attacks on service providers and individual organisation's web sites, is due to network complexity. Millions of smart devices interconnected. All have bugs in their software making them vulnerable, and all capable of random hardware faults and glitches. Mixing film quotes, failure is not only an option, it is an inevitability. The manner in which hardware frailties and software bugs are detected and dealt with prevents them from becoming catastrophes.

The problem is complex because of different types of networks and many different types of devices, so testing every combination of everything in all possible circumstances is impossible. However this should not be an expedient excuse for insufficient testing in the rush to get products or services to market, and tests should be performed at each stage of deployment, with ongoing monitoring thereafter.

What types of testing are employed in networks of smart devices? Most often tests are performed on individual units—functions in programs, mobile devices, components in a network—with further testing on a larger scale when these are integrated together. Finally, applications may sometimes be tested by users, but how many organisations really exercise the full functionality of these component devices or run continual tests to fully exercise the performance of their networks?

Simply employing the standard testing procedures of exercising each function or product in isolation, and then again through the integration process may not be enough. Although many systems are exercised and tested pre-deployment, this is unlikely to cover all of the possible scenarios, especially with increasing sophistication and interaction. A further stage is necessary; active testing on the whole system is the only way to prevent the out-of-the-box, impossible-to-occur failures that seem to be increasingly frequent. Only then can systems meet their performance and service quality goals.

Whole lifecycle testing might not be glamorous, but it can be valuable. Why? Well, as has been long proved in software development, flaws caught earlier on are always cheaper to fix. For technically sophisticated products that is important as it may prevent recalls, post deployment patches and expensive tracing of convoluted interactions, but the greater cost saved may be far beyond the technology. Consumer confidence, brand leadership and customer satisfaction are all valuable and take time to build, but can be undermined by a simple undetected bug.