What's The right Number?

or

4 stages from disbelief to acceptance

Originally published on January 17 2011, 2:53 PM

 

As you probably know, my “other” job is being a Business Intelligence consultant.
I’m old enough to position myself also as a management consultant in the area of control and management accounting.
Actually, I range from DBA to management consulting, with anything in between. In my projects, one of the things I often do is replacing old, unstructured, control systems with something more organic, based on a datawarehouse and a dedicated reporting system.

One of the issues I meet in these projects is what I call the “Match the figures” syndrome.
When people rely on incomplete data sources, like the reports returned by transactional systems or manual notes taken on Excel etc.; they are so concentrated in gathering and organizing their figures that they overlook data quality. For data quality, in this case, I do not refer to the classic issues of duplicated customer addresses, but to the strange patterns that can be inside data. These are originated in two cases. There might a be technical error, with incomplete data, inconsistent codes etc., and Excel swallows the most of them hiding the issue to the user. There might be unexpected data patterns that are hidden inside aggregate data but they become relevant once you have the details available; for example, for a customer adhering to a 3x2 promotion, you could give the purchase of the 3 items for granted while the truth is that some just bought two.

Once this syndrome surfaces, then the customer climbs a ladder from hell to heaven (or simply fires the consultant in the process, either of two will work, somehow).

The first stage is simply “the new system does not work”. This is the first reaction and is countered by the hard facts. Just show how each and every transactional record required is there, with the right figures and the right values in fields. This may not be trivial, if some heavy transformations have been done, but it usually works. It also generates the question “So, why the figures are different?”.

The second stage is “the system works but needs a lot of trimming”. And, indeed, it requires trimming because all the implicit assumptions that have been made to create older figures must be worked out and implemented in the new system. Often it is a painful job but, in this stage, system trust builds up. I put the burden of trimming on the users’ shoulders: they get familiar with the new system and the new data.

The third stage is “We did not know that”. Users realize that they have the data quality issues mentioned above and the first reaction is “It’s not my fault, data are not mine.”. Of course is not the analyst who caused the data quality issues but the data underneath are generated inside the company. If it’s a tech issue, the IT must be aware and possibly fix it. If it’s an operational issue the management is not fully aware of (yes, this happens more often than you think), than a decision must be made on it, whether to keep doing things this way or not.
In this stage an objection often rises: “We only needed a better insight and some new data, we did not mean to reorganize the company or involve anyone outside IT or controlling”. My reply is usually “So, you wanted a better insight to do nothing with it?”. That’s the point where the consultant is most likely to be fired. 

 

If I’m not fired, the fourth and final stage is "system acceptance", where the system is progressively rolled out and its use spreads among the information workers. Everybody is happy until some other issues rise.

Lack of flexibility (“You take one workday just to add a f*** table that I already have in Excel?”), the effort to build a system just to have few numbers the top management already had before (“ok, you’re telling me that I have a 0.12% gross margin overestimate, so what?”), and, most of all, users love Excel because they can control each and every value while, on classical systems, they do not feel to be in control anymore. So, usually, pros and cons are somewhat balanced.

Anyway I think that the deeper insight that you can get from a classical implementation is not tied to it. If you take your ERP system reports for the absolute truth, and you never verified how those figures are derived, you are blind to some phenomena which are hidden within your data. If you have a consistent verification process in place, you can reach the same point as a classical system can take you to. 

So, what is the main point? There’s much more inside data than you usually think, but this is not a technical point. Data are yours, you create them, you control them. It’s up to you to exploit them at their full potential.