Taking the complex out of complex event processing

The quote of this week is taken from an article in InformationAge that talks about operational intelligence. 

The article explains what operational intelligence means, and you can read it to see if you find anything new.

The point of this post is a quote done by Ivan Casanova from TIBCO:  

“We should all be focused on taking the 'complex' out of complex event processing" 

This quote is in the context of explaining the acquisition of Streambase by TIBCO.    I don't know Mr. Casanova personally, but what I have learned from his statement is that he believes that going forward, the programming model and tools represented by Streambase are better fit and less complex to use that TIBCO has done before, where it extended RETE based business rules system to handle stateful event processing cases, while retaining the rule-based programming model.    Streambase is using an "event flow" model that is some variation of event processing network.    Without getting to analysis of specific products (a restriction I have taken upon myself in this Blog), I would say that overall I believe that as a conceptual model for event processing I believe in the EPN model (which is of the family of data flow models),  and in visual working environment (better than textual working environments) to design and program.   This reduces the complexity for IT developers, which I think is very important trend.   The ultimate reduction of complexity requires one more step -  event processing modeling in the level of the business user level and automatic translation to an implementation language.  

Bottom line: I agree with the statement in the quote -- actually this is my main area of interest nowadays. 

Statistical reasoning and event processing tutorial

Streambase has posted a video tutorial on combination of their EP product with statistical reasoning based on MATLAB.  The idea of combining event processing and statistical reasoning is becoming part of the big data offerings, and no wonder that statistical reasoning vendors are adding event processing to their portfolio, for example the introduction of event processing within SAS, 

Streambase takes it from the other side -event processing vendor that combines statistical reasoning.

Interesting tutorial to watch.  

Start small, succeed and then scale - Mark Palmer's best practice on employing event processing in an enterprise

One of the observations is that it is not trivial for enterprises  that are used to legacy systems and legacy systems and legacy developers to adapt to event processing thinking.  I have written about it recently.  A recent meeting with a customer triggered the initial reaction - "but we don't know how to work this way", the way they know how to work is: getting events, inserting them to a database, and ask periodic and on request queries.   I have also recently compared the request driven (legacy) approach and the event driven approach. 

Mark Palmer, CEO of Streambase, has recently written a "best practice"  on this topic of how to introduce event processing in an enterprise, and entitled the best practice "Start small, succeed and then scale".  

Mark starts by asserting that real-time becomes a must do agenda in many enterprises, and many type of applications, but the problem is that both legacy systems and people require an overhaul. 

His methodology is - choosing a small, but important application, put the best people on it, iterate if failure, learn from experience, and then scale it to other functions and applications.    I think that this is a good approach, it would also be interesting to have some experience reports about application of this best practice in reality.     I'll write more about this topic - later.

Richard Tibbetts on "visual programming for complex event processing"

This picture is taken from the first slide of the presentation given by Richard Tibbetts last week in Oreill'y  OSCON Open Source Convention.   The presentation is now available on the web.  


In the presentation Richard brings five lessons for the implementation and use  of visual programming from Streambase experience.  Note that the visual programming in this case is intended for developers, building visual programming for domain experts is a different saga.   
In my event processing course, students have a choice to do projects in various available platforms (including implementing everything hard-coded),   I found that many of the students prefer this style of visual programming, but some prefer to work with text-oriented programming that looks more like the programming they are used to, claiming they can have better control that way, I guess from the same reason that there are people that still prefer to drive manual cars.   Personally I think that visual programming is the way to move forward, but we still have a challenge to do visual programming for semi-technical people.