Subscribe to our Feed

The Expansion and Contraction of “BI”

Posted on Saturday, Apr 26th 2008

By Dave Hatch, Guest Author

I have now completed a full year of research at Aberdeen Group (the last 6 months have been the busiest of my career, by far!), and I can truly say that after 20 years as a user and marketer of BI technologies, the past year has taught me more (as a researcher) than the previous 20 did while I was immersed in the day-to-day struggles of the software environment.

After conducting 10 major primary research projects, and benchmarking over 5,000 companies, one very interesting “trend” has emerged: BI’s “expansion” – to more end-users, and addressing more internal and external data – is being pressured and somewhat stymied by BI’s “contraction” – the need for simpler, less costly, lower resource-dependent solutions that address information needs closer and closer to real-time.

My research has shown that companies are feeling pressure to expand the use and accessibility of “business intelligence” (reporting, analytics, dashboards, scorecards, etc) among various groups of end-users within their organizations (and to external stakeholders such as customers, partners, investors, etc). The challenges to accomplishing this are many. In fact, survey respondents identified three factors that are inhibiting BI expansion within the enterprise:

– A lack of BI skill sets among non-technical business users
– The inability to integrate data from all sources necessary to meet business needs
– Poor data quality – end users do not trust the information

Moreover, 80% of all companies surveyed rate the delivery of actionable information to the enterprise as a top-10 priority, yet 37% report that enterprise use of business intelligence applications has stagnated, decreased, or that they do not know which way usage is trending. The reasons for this are numerous, but the conflicting forces of expansion and contraction are at play, and this is really what BI technology solution providers are tasked with solving:

– Increased access to actionable information – to new non-technical stakeholders who have been underserved in the past
– Improved delivery of actionable information – via a self-service method to the enterprise requiring little or no IT intervention
– Decreased “time to information” – to accelerate decision-making and actions, the gap between business events and the availability to information about those events needs to be shortened

As companies continue to drive BI’s reach to non-technical users, particularly the line-level knowledge workers who have traditionally been underserved by BI technology, they are also seeking new methods for delivering access to BI that alleviate some of these challenges. Aberdeen’s research uncovered a significant growth in interest among respondents in new methods for BI delivery. 39% of respondents state that they plan to access BI as an embedded capability within enterprise applications, and 31% report plans to access BI capabilities via “On-Demand BI”, or BI delivered within a “Software-as-a-Service” (or SaaS) offering.

Research also reveals that interest in external and “unstructured” information, such as data contained within Web 2.0 applications (instant messaging, web pages, blogs, wikis, RSS feeds, etc.) and other customer-facing applications (web page – content management systems, email, customer service applications) are increasingly being targeted as important sources of information and functionality for future BI solutions. 46% of respondents plan to integrate Web 2.0 content with existing BI reporting and analysis applications, and 44% report plans to tap into web content management systems and web pages for additional external BI content.

Unstructured data is found primarily within the consumer-driven applications that a new generation of knowledge workers have grown-up with and are now incorporating (whether management likes it or not) into the day-to-day mix of office productivity tools. Companies must begin to recognize the potential power that can be derived from a strategy that leverages the natural progression of web application technologies and the user-created data and “group wisdom” that can be captured. But this also increases the “expansion” of BI beyond current growth rates as these new sources of data represent 3 to 4 times the data volumes that have traditionally been addressed by BI. So, if data quality is currently a major inhibitor holding back the expansion of BI… what does the influx of ungoverned external do to the equation?

I will continue to research the conflicting forces of expansion and contraction as it relates to business users’ quest to obtain access to trusted information faster and easier… I would be interested to learn about your organization’s approach to dealing with the expansion and contraction issues. Feel free to post responses!

Hacker News
() Comments

Featured Videos


Dear Dave:

We have been involved in web based global manufacturing execution systems for several years. A lot of that is finding and ranking actionable situations that need human attention. This was mostly driven from a complex data model.

In the last two years we have developed a
patent pending (a real patent, not just a memo
to the USPTO) NLP engine to analyze and
organize unstructured data so it can be understood in the context of a business execution environment.

This unstructured data is non-trivial to analyze
and keyword searching, Lucene indexing etc.,
LSI are not going to give results that a typical user can use.

Anyhow, if you are interested in finding out more,
please contact me by return email. We are current;y putting together a web and DVD video series that explains both the theory and the practical use of this engine.

BTW, over 30 top researchers from around the world contributed/reviewed directly the design
of the system, and concurred that it was a valid approach. These experts were from AI, cognitive psych, neuro science, applied math/pattern recognition, NLP, linguistics.

Anyhow….thanks for your attention.


Steve Kohler
678 947 6857 U.S. E.S.T.

Steve Kohler Saturday, April 26, 2008 at 7:03 AM PT

[…] Blogroll, Learning, Model, Motivation, Strategy — Subbaraman Iyer @ 9:44 pm Dave Hatch writes a good report on the expansion and contraction of the "BI" […]

State of the Business intelligence industry « Serendipitous moments Sunday, April 27, 2008 at 7:16 AM PT


Good assessment and I more or less agree with most of the issues that you have highlighted. I have expanded the scope of the issue and provided more insights in my blog post Aat:

Subbaraman Iyer Sunday, April 27, 2008 at 7:18 AM PT

David and Subbaraman,

I work for an Indian IT services MNC, and we work with all the top vendors of BI products to implement the solutions for customers across multiple verticals.

While I agree with you that lack of BI skillsets (especially ability to create meta data models) could be one of the reasons for lack of RoI on implemented BI solutions, I suggest you also consider the ‘domain knowledge’ (or lack of it) of the product vendor and the system integrator.

I feel that BI products today are very difficult to verticalize. That is, an Analytics product can not be implemented with the same success in say Pharma, as it could be in Retail. The reason for this is that product design is not very fluid to take into account the varying challenges for businesses across verticals.

However, this can be/should be handled through customization, by the product implementation team, which comprises the vendor’s product experts (who know the product very well, but not the domain), the system integrator’s engineers (who neither know the product very well nor the domain), and finally the customer’s business analysts (who lack the required IT understanding).

If we get a product that is flexible and has product specialists with deep domain knowledge, plus IT services vendors who can go beyond basic system integration and add real value through their techno-functional knowledge, and then finally add to this mix a set of business analysts who can pinpoint the data needs of end users, there is no reason why BI can not deliver the promised value.

Sadly, my experience with developers from my own company (and some competitors as well), and with product specialists from the BI vendors, shows that most of the heavy lifting is expected to be done by the customers’ team of end users. And the project is a success to the extent these business users can communicate with the technical team or vice-versa.


Kumar Narasimha Sunday, April 27, 2008 at 10:00 PM PT

[…] The Expansion and Contraction of “BI” – Sramana Mitra on Strategy […]

Chapter 5 | Offshore Welders Thursday, January 22, 2009 at 9:40 AM PT