Sramana Mitra: Let’s double down on how a data warehouse becomes an information warehouse, given what you are seeing.
K. R. Sanjiv: Data itself is of limited value. Today the focus of an organization is to assimilate all the data and have data that is reliable, comprehensive, and consistent so that it can be used across [areas].
That itself is a big exercise, and half the organizations are struggling with it – between an application that requires a business user to understand what offer should be given or what action should be taken on a particular customer, so that his experience with the organization improves or revenue increases, there is a whole gap from the data to that application – there is a set of activities to be done.
Today it is done with a hypothesis: “Let me try this, then you look at the data and you determine if giving the customer an offer he will like it and we should do that.” With the kinds of volumes that are coming, the latency required, and the complexity and amount of data available, that is not optimal usage. You will run out of scalability. So, an information warehouse is something which you will define for a business unit, just like you define a scheme for a warehouse. For example, a withdrawal from an ATM that is twice the average – that is information. Or five telephone calls to a telco provider requesting a SIM change is also information. Just as you model data, you start modeling information that can be used by applications in real time. It is a combination of business rules – x + y under z circumstances – a combination of statistical models that will mine the algorithm and the raw data that come in from the data warehouses. Combining all these, you get an information repository. That can be used by [consumer] applications.
SM: I assume you are doing both exception tracking and pattern tracking. Is that correct?
KS: Yes. If you look at it, there are exceptions that are pattern related and there are events that also need to be tracked. So, there are exceptions, patterns and events. A change of address, for example, is an event. A change of address can mean that now the person needs a new house loan. That is information as well. All three need to be tracked.
SM: Tell me about some of the emerging trends. What is really interesting, exciting and cutting edge?
KS: The chain that I explained is a major disruption. I don’t think many organizations are ready for this. It is complex and it requires a lot of innovative systems that have to be designed and engineered. I think from an entrepreneurial perspective, there are a lot of opportunities. How do you build self-learning systems? How do you go from the ERP systems and back-end systems into an analytical system in real time? If I am running a trade promotion, how does it work? If I am applying analytics to equipment, like a blood infusion pump, for example, how do I interface it with a downstream system and feed it back so the performance improves?
Entire machine learning algorithms have to be put in place, which is definitely one area where innovation is happening, and we are also investing in that. The second big area is visualization. Analytics has become much more real time. There is machine learning which improves analytical models on a consistent basis. Visualization is going to change drastically as we move forward. It will no longer be the conventional BI applications. Visualization will be much more discovery based. You can think of a combination between a structured query and a Google search, which will come together as an integrated interface. It will be much more user driven and visual. Innovations are happening in the layer of visualization.