I recently came across a worrying set of market figures from industry analyst Nelson Hall that estimated the software testing services market was worth $29bn in 2007, of which $6bn was spent on "independent testing", where testing activity is outsourced and typically off-shored. These estimates then predicted that the market would increase to $37.6bn by 2012, with the market share for independent testing growing to approximately 30% ($12bn).

I was immediately very concerned by this growth and began to wonder why businesses were choosing to invest a greater proportion of their test effort independently from the projects producing the software. Surely this will only result in software issues being discovered later on in the project, when it is exponentially more expensive to fix them?

The pitfalls of outsourcing testing

More businesses are moving towards this trend as they are primarily driven by the cost-savings offered by these out-sourced services. For example, most test services providers are known to offer double the testing resources in India for the same price as independent testing performed in an on-shore location.

Particularly in the current economic climate, I completely understand the desire to reduce costs, but this sounds like a false-economy. The real value from adopting an outsourced test approach has to come from a measure of the total cost of adopting a focus on independent testing, as opposed to one of investment in embedded testing where test activity occurs within the time-frame of other software development activities in the project.

Recent thinking within the software industry and common sense suggests that removing test activity from the original development site and team never makes the delivery of an application more efficient, of higher quality, or lower cost. In fact I believe it's quite the opposite. Failing to have test specialists as an integral part of a team, who understand the latest team position, test regularly and often, typically results in an overall decrease in quality and a larger set of defects that need to be addressed by the project team late in the project lifecycle, when it is far more expensive to resolve them.

Hence, whilst it may appear cheaper to separate and "industrialise" all test activity into a reactive test factory that processes the outputs of a development team after the application has been developed, the total cost of delivery success will increase substantially.

The temptation to adopt this outsourced testing approach also encourages a more linear, often referred to as "waterfall", approach to software development. This approach is widely recognised as a very poor delivery model, typically resulting in an increased risk of project failure, delayed delivery with an associated cost over-run, and a system that is unlikely to reflect the changing needs of the business.

However to move the development approach toward a more iterative or agile style, again requires greater integration of test activity into the development team.

But, why is outsourcing software testing such an attractive prospect?

It is worth noting that the idea of outsourced testing services isn't all bad, plus it's easy to see why businesses opt for the outsourced testing model. Yes, you can establish a highly efficient, low cost, leading practice, independent and flexible test resource to provide a high level of quality assurance prior to a system being released into a live environment. There is a lot of value in this service, and it is important to have a "last line of defence" in place for your business operations, but it should not be confused with the need for testing as part of the development lifecycle.

Testing as a discipline within the software industry is changing dramatically. Gone are the days when testers were seen as the team brought into projects to give a final assessment of the quality (or more typically the poor quality) of a system, prior to its deployment. Leaders in the industry now embed test activity at every stage of development, ensuring that test expertise is placed within a collaborative development team, and that tests are performed on any aspect of the solution the moment it is produced.

Testing as a profession is asking test practitioners to step out of their "safe" test communities and become part of a wider software development team. This calls for a wider range of knowledge and skills, but will typically be more rewarding as they become part of the success delivered by the team.

Therefore, those who decide to use an outsourced testing service need to recognise the mix of testing solutions required to make project teams a success. The "one size fits all" approach of isolating test activity to a remote and reactive team is only part of the solution.

However, in an economic environment where many organisations want "more for less", the offering of outsourced test services to replace expensive full-time test resources will always appeal to those who don't appreciate the big picture and have been given the "hard-sell". When that market segment is also seen to be growing it will quickly become a target for those offering these services, even if the end result for the client may not quite be as they expected.

So, whilst I expect some agreement from outsourced test services suppliers around the "ethical" nature of encouraging greater investment by clients in their embedded test activity, the financial opportunity for these suppliers may be so great, that I also expect my complaints to fall on deaf ears. I suspect that this could be a case of market forces winning over ethical and sensible practices, but if the above considerations are taken into account by any organisation looking at outsourced test services, then hopefully a balance of approaches will be reached as part a wider test strategy.