On two-tier analytics

Will Cappel from Gartner has written about two-tier analytics and went back to Immanuel Kant (in the picture above) as support to his thesis.   Kant argued that the human cognition work in two levels:  the first level that grasps objects and raw facts about them, the second level which captures causality between these objects over space and time, applying some levels of simplification to what Kant said, he is right.  Cappel makes the analogy to the analytics world, and says that the first level is satisfied by event processing that process events by filtering, transformation and pattern detection to identify higher level situations.  The second level is satisfied by pattern discovery engines that work on top of the first level.    This is an interesting observation, I think that the picture is somewhat more complicated as there are more tiers.    Event processing detect patterns in real-time, and indeed one of the ways to obtain these patterns are the pattern discovery mechanism over historical data, which may include the results of event processing systems, but should also include many other data items that describe the impact on the environment,  since situation detection triggers actions, and actions impact the environment,  the pattern discovery needs feedback from the outcome along with feedback from the process itself.    The interesting part comes when we add real-time adaptation to the picture, here, in a similar thing to how the cognition works,  the causality relations may change on the fly.   Consider traffic management systems, studies show that these systems are chaotic in nature and one cannot forecast patterns of behavior based on past experience with sufficient accuracy, forecast is limited to about 15 minutes to the future in some cases and the control policies for highway should constantly adapt.    Here we need four tier analytics:

The first tier is the off-line tier which change the setting of the system based on historical learning.  

The second tier is the event processing tier which observes and monitors

The third tier is the real-time forecasting tier which adapts the causalities and make the short term forecasting

The fourth tier is the real-time decision making tier which makes the best decision possible within the time frame allocated for the decision (which may not be the global optimized solution).

Bottom line:  I agree with Cappel about the multi-tier approach,  and pointing out that reality is somewhat more complicated...