Engineering The Information Flow

The Role of User Generated  Content

In many organization, even those with a developed BI landscape, the notion of engineering the information flow to support processes and decision is not entirely understood.

What I am going to describe in this post is how processes stimulate the growth of the organization's information assets and the healthy life cycle to manage it.

How the information process is born

It is often thought that the process of providing information through BI is quite straightforward: users have requirements and BI professionals/analyst satisfy them. Since every answer, in this environment, is just the beginning of another question, then the loop will start again. In this way we have simply moved the point of use of BI closer to users than when it was with IT. Self service BI tools, now widely in use, reduce the number of cycles but just for simple and quite standard interrogations.

In a more modern and structured environment, though, every group of users is provided with a set of reports, datasets, dashboards or interactive reports (and any other form of consumption you can think of) that covers the entire breadth of the information available that pertains a group of users. Users will chose the information they need from the information available in a comprehensive "conceptual" menu. This is obviously the environment where self service analysis can thrive and prosper.


While the organization processes evolve, the information process supporting them evolves too.
In a structured environement, users will start from the information available, will "build" something, "hack" together something else, add some user generated data to get some results.

For example, a marketing department may produce, on the basis of available customer data, a model to increase sales in a price-competitive market without denting margins by pushing the customers to a different purchasing mix. 
At the beginning this model will be "on paper" and it will not be integrated with any system. It will be put in practice with ad-hoc operations and tested to be effective. 
Once the effectiveness is confirmed, it will become part of standard procedures, part of a process or a process of its own. At this point, the model must be engineered to be part of the information provided to actively support its development. Following the same example, the model might require to e-mail the users eligible automatically and e-mailing campaign result should be made available to the BI users. 
What was calculated and reported once on an ad-hoc basis, now needs to generate automatically customer attributes to be mapped in semantic layers, product discount schemes available for analysis in standard tools and feed those data to the transactional systems that operate the scheme.

The cost of not engineering the scheme (notice that it would not have been done according to the traditional view until the users explicitly asked for those data being engineered) would be an increase of manual operations, information entropy and headcount to manage it. 

What is described here is a healthy cycle where the business initiates a class of analysis and than it is engineered and implemented in the main flow and becomes part of the organization's information assets. 


The enhanced information assets are now part of the information flow and their benefit grows beyond the intended purpose they were created for. They may become part of the factor composing other departments critical information assets or high level business views.
Sticking with the previous example, the execution of the initiative may be immediately known by the shipping department, which is downstream in the cycle and will be required to fulfill a different order type.
The same information may become part of the revenue drivers analysis, which is, generally, a finance and executive domain.

The Paradigm Shift

This probably doesn't look particularly revolutionary, after all it is basically good sense. However, there is a paradigm shift hidden in the approach described above.

In the classic approach, information sits somewhere until someone thinks that it may be useful. In the approach described now the information provided to users is all the information the need and ask plus all the information they will likely need in the future. It is part of the BI professional job to identify which information might become useful in the future for which users.

Part of the extra information  is surely going to be derived from the classical sources, but as the complexity of the overall system is increasing, an increasing amount will be the result of the engineering of users' analysis. Every addition provides deeper insight and more business specific decision making elements.

As a consequence, BI systems tend to start with simple business models that shape the information content, and evolve toward an increasing complexity, and, potentially, a better decision making instrument.

See you next time!