Low Environmental Impact Business Intelligence

Originally posted on 2/26/2009 

Photo courtesy mangpages

Low environmental impact business intelligence


The title will probably let you think that I’m following like a lemming the current buzz on green technologies. While I’m seriously concerned about the future of our planet, the environment I’m referring to is your company environment.

Usually a business intelligence initiative is a huge project.

It requires a large amount of abstract thinking and a thorough analysis to define a sound architecture.

It requires a wide range of professional skills to develop the technical infrastructure.

It requires the involvement of many corporate levels to make it work and spread.

Even when the road of small incremental releases is taken, there are plenty of them in quite a long period.

The technical effort behind the project often appears disproportioned to the rest of the company, notably including the upper management. Seen from the brass, the task of collecting few relevant numbers appears to be trivial, unless there has been a specific experience in the subject. The overall BI initiative may well appear like building a castle in the backyard.

The impact on IT systems may be even more shocking. Often the company IT department is primarily concerned about transactional systems operations. That’s natural because of their inherent critical nature to the business.

The BI technical requirements are then seen as a pain. New servers, new users, new unfamiliar pieces of software invade the server rooms. Worst, they appear to colonize each area of the company, sucking data from and providing nobody knows exactly what (“Why can’t I run a report on my transactional database?”). Data stewardship initiatives start touching tables that were left untouched for years, thus becoming a "company mantra". Suddenly separate systems become inadequate as they cannot match each other data in some obscure analysis.

The resulting information earthquake contributes to making the BI guys look like elephants in the glass shop.

Thus, the price paid for implementing a pervasive BI system may be high; disproportionately high from some perspectives.

So, I’m asking myself whether there is an alternative to this approach.

The answer is yes.

Where BI is yet to be introduced, often data are equally available. They take the form of printouts, text files, pdfs or Excel. They are inherently incoherent and fragmented. The users merge and integrate them manually, and the main tool for doing this is Excel.

Users are unaware of the potential of a properly designed and regularly updated datamart, so they perceive their work as the only way to gain data awareness. While doing so, they implicitly perform data quality (“Humm, this order is a dummy one, let’s cut it off from the total”) and rounding. Equally they enforce data consistency making assumptions on how to calculate measures according to specific needs.

So, while a large effort is spent on generating spreadsheets, the result is inherently aligned with business needs. This is what should be saved from this manual process, as the corresponding engineered process is often hard to implement.

Thus, maybe, the best thing we can do to ease information workers' workload and start spreading the BI verb, is simplify their daily job.

First, we should find a way to automate data collection from the systems. Not much can be done better than current Excel or MS Office wizards do, on this front. Many transactional systems generate MS Office compatible files.

Then we should provide a way to save the data extracted to a central repository, in order not to have them diluted on numberless files. Unlike traditional systems, the user himself should be able to define the data structure and change it whenever he needs to.

Then the user should be provided the capacity to perform transformations on data. While the Excel sheet is still the best place to do the job, some automation may be welcomed. Transformation results should be saved to the repository too.

Last, data should be queried from the repository without any technical knowledge and placed inside users spreadsheets with no formatting constraints.

This kind of BI speaks a language much closer to the user, so can gain active support more quickly. It is a concrete effort to ease the user workload without working indefinitely on baroque technical architectures.

Being much simpler, it is implemented quickly, lowering the project risk for the customer.

It uses tools already familiar to the user (Excel was used as an example, but whatever else from the stack is appropriate) with obvious benefits, letting the user concentrate on what matters most, the data analysis.

I do not think that this approach can displace traditional BI, as it has many intrinsic limits, but it has a niche for sure in the corporate world.



164 views and 1 response

  • Nov 29 2010,  4:37 AMHiran de Silva responded:Gus, I agree.Most business activity in offices involve a combination of Excel, Word, email, importing and exporting from enterprise systems, telephone and conversation (I read somewhere a figure of 95% on some Gartner reseach on BI alone).The people best positioned to improve the productivity of these activities are empowered users themselves. They are the most motivated, most knowlegable about the processes, and have the most to gain from productivity enhancements.There is a misconception that anything that involves IT is the responsibility of IT to do. That is ludicrous. If IT have to improve 'the 95%' the IT department will need to increase by 20 times (!) - and then IT will be the biggest department in all organisations, and the business will exist solely to support the IT department!So, yes. Anything that will help the users become more productive, make the processes they have already designed more efficient, extend the tools they have already been given, and extend the training they are given will add value to organisations.I have come across some IT people who feel that the trend towards self-service will harm their job security. I don't think it will. As I said, their real job is to keep the technology infrastructure working and that is not affected, indeed their skill will be needed even more, because the users will be making greater use of the available technologies and those will need to be supported even more.