Tuesday, 29 December 2009

Does data quality matter in an MDM solution?

Data Quality is often considered the panacea of a successful MDM implementation. However, in my experience, any attempt to achieve good data quality across a complex enterprise is almost certainly doomed to failure.  Good MDM solutions therefore, have to work with inconsistent and poor data quality.

If there was a single industry data quality standard that applied to the finance and insurance industry, what would it look like and how difficult would it be for the average UK FS&I company to conform to the standard?

This is a probably the minimal requirement for such a standard:

1.  a set of mandatory attributes should be captured for each individual sufficient to uniquely identify them. [e.g.  title, name, sex, date of birth];


2.  the relationship each individual has with the company should be captured using standard terminology. [e.g. share holder, product owner];


3.  the role each individual has with a company product should be captured using standard terminology. [e.g. life assured, beneficiary];


4.  any change to individual circumstances that affects any of the standard attributes of relationship with company or products should be captured immediately. [e.g. change of name, change of address];


5.  any change to the status of the relationship between company or product and the individual should be captured immediately. [e.g. at product maturity role changes from Life Assured to  Maturity Paid].


In reality:


1.  the only absolute method of uniquely identifying an individual is probably to use their encoded DNA or finger prints.  However, this is impractical to implement on a large scale, and even these scientific methods are challenged by some within the legal profession;


2.  even if the industry could agree a single terminology there would also need to be an agreed set of codes and abbreviations to facilitate IT storage requirements;


3.  there is no obligation on 3rd parties to pass on changes to individual circumstances to interested parties, even if the interested parties were known, and the disclosure did not contravene data protection rules;

There is an additional set of attribute that should probably be captured in support of the products where the individuals holds a significant role..


1.  a standard set of attributed for each individual / role combination [e.g.  for a Life Assured role – health status [smoker / non smoker], proof of data of birth etc]


2.  any change to a 3rd party relationship with the individual that affects the mandatory attributes or the relationship between company / product and the individual should be captured immediately.

I don’t think anyone would seriously suggest such a standard could be attained within a commercial organisation.  Yet time and time again, data quality is put forward as a fundamental first step to a successful MDM implementation.

In my view the facts are:

1.  each business has different needs in the area of data capture and the cost associated with the capture of non product specific data is not sustainable.  Any MDM solution therefore has to cater for different data sets for different product / role combinations;


2.  customer data can easily get out of date when the product life cycle does not necessitate regular customer contact.  For some products, contact may be at the start and end of the product term.  In some cases there may only be contact at product inception – term assurance for example where the customer dies during the product term.  Any MDM solution therefore, has to cope with out-of-date data sets;


3.  change of customer circumstances over time is a certainty and can vary from very predictable events like marriage, to unpredictable events like change to financial arrangement.  Any sensible MDM solution therefore,  can’t rely on the maintenance of up‑to‑date concurrent data.

If we take these facts into consideration, the obvious conclusion has to be that any attempt to maintain a consolidated master view of customer data can not be based on the concept of data quality standards applied enterprise wide across an organisation.

A successful, MDM implementation should build on and complement the product based data sets.  To achieve this, more complex methods of creating and maintaining a consolidated view of Master Data need to be employed that do not rely upon data quality standards.