Sunday, December 30, 2007

Event Processing for Business Intelligence



From the various descriptions of Business Intelligence found on the Web, I have chosen the one by Hakki Erbug as a starting point.



As noted in several previous postings, IMHO, event processing is a set of technologies that have multiple usages, and are not really strongly associated with a single type of application. Business Intelligence is similar in the fact that it combines various technologies, but different in the fact that it is focused around area of using data for decision making in various ways. In the event processing case, decision making is one of the possible usages, but not the only. This posting will briefly survey how event processing can enrich business intelligence, and in fact, in the Gartner EPS summit I have seen several BI vendors that are looking at EP as a natural capability to complement their products.



Going on Erburg's illustration clockwise:



1. "Active Data Warehouse" - While traditional warehouses are being updated in batch, the notion of active data warehouse makes the warehouse update itself an event-driven action. The rationale is: when a certain event (raw or derived) occurs, a decision has to be taken, however, the decision relies on a data warehouse, thus, an update of the data warehouse should occur before making this decision. The update can be of the same event that happens or of some collection of data that has still not been updated in the data warehouse and is needed for the decision making. There can be some time constraints associated with the decision (and in turn with the warehouse updates). The time constraints are not necessarily micro-seconds, the constraints can be minutes or hours, but they are typically well-defined.

2. ETL and mediated event processing: ETL has some functional similarity with mediated event processing, it is also about transformation. We see mediated event processing and ETL getting closer to one another, where difference may be in quality of service. In the future there may be a case that ETL will become a specific case of mediated event processing (of course ETL folks may say the same from the opposite direction).

3. Real-time analytics: While analytics (simulation, optimization, mining...) has been used for a while in the decision making part of BI, in the event-driven world, the reaction to an event in some cases is temporally-bound, which means that there is a real-time constraint, or upper limit on the requested reaction time. This provides new way of thinking about analytics - while without time constraints an optimization should strive to get the "best result" (or if heuristics satisfy some approximation condition), in real-time analytics the optimization strives to get "the best result that can be obtained in T time-units as specified (e.g. 18 seconds)". How does event processing play in real-time analytics? it may play in a simulation mode - scenarios are created and simulated events are emitted - they in turn may create simulated derived events which determine the situations of this simulation. This is, of course, in addition to the fact that in an event-driven universe, the entire BI cycle is event-driven and relates to the event and its context.

To conclude -- event processing is a natural step in the BI capabilities, and thus I expect BI suites to support the event-driven flavor... This - again, does not say that BI is the ONLY use of event processing. I still need also to refer to the issue - "has BAM failed because it was not based on BI techniques" as claimed in the article that triggered my discussion on the BI topic - stay tuned.

1 comment:

avi said...

Event-driven real-time simulations could be in the shape of:
"every 10 minutes, create a simulation based on t-10min:t-20min". If the job takes less than a minute, there shouldn't be a problem to decrease the time parameters above.