The data quality market is a surprisingly fragmented one for a market that is far from new and trendy. While there are a few leading vendors, there is no dominant one, and I have identified over 60 software vendors selling data quality software. I think there are a couple of reasons why the market has not grown more than it has. One is that many (by no means all) data quality vendors focus on providing a tool with a clever algorithm for, say, matching records or cleaning up name and address lists, yet pay little attention to supporting the governance processes that lie behind the causes of the poor data quality in the first place.

As well as measuring the state of existing data and fixing up errors, it is important to also understand the business processes that support the lifecycle of that data, from its creation, update and even retirement. The reason for poor data quality may lie in unclear roles and responsibilities, duplicate or conflicting processes in different departments. There may be a lack of understanding by one set of business users as to why certain data attributes need to be recorded (perhaps for the benefit of another part of the business). Data quality projects need to address these process issues if data quality is to be improved structurally, rather than being just a one-off fix.

Another issue is that many data quality tools specialise in dealing with very specific data to the neglect of others. The obvious example is customer name and address, but other vendors specialise in dealing with, say, product data or spares data, but little else. While there is certainly value in fixing up data of one type, the business as a whole may get more benefit from tackling a broader base of data, for example dealing not just with product data but the locations that these products are delivered to, and the customers they are supplied to. By tracing the lifecycle of data and mapping this to business processes, issues may be highlighted which cause data quality problems. In the absence of vendor attention, systems integrators are filling the void.

An example of this is Chicago-based Utopia, who, though only founded in 2002, have built up a fast growing business specialising in data lifecycle management solutions and consultancy services, especially in the process manufacturing and consumer products industries. They have combined a deep understanding of data with a vendor-agnostic approach.

The company has also created libraries of useful data-related tools, including data governance management facilities, which they use to integrate broader data lifecycle applications utilising specialised data quality and master data management products. With 160 staff and 50% revenue growth last year, Utopia is prospering in a way that few data quality vendors are. Many data quality vendors could learn some lessons from this and consider how to broaden their offerings to encompass support for a wide variety of data types, and also support the business processes that surround this data.