Call for papers: special issue of ACM Transaction on Internet Technology on "event recognition"

ACM Transactions on Internet Technology issued a call for papers for a special issue on "event recognition".  In the CFP they explain that "event recognition" mean "event pattern matching".  This is an opportunity to report on interesting research work in that area, with a relatively short review and publication cycles.

On the OODA loop and the 4D

Richard Veryard asked in a comment to my post on  the recent tutorial given by Jeff Adkins and myself if our 4D scheme (which was introduced by Jeff)  is related to the famous OODA loop.  I have mentioned the OODA loop in the past in connection with the need to act faster than the speed of thinking. the strategy of air combats, later he also made claims about the generality of this method.    The 4D is certainly from the same family, and the four stages are indeed similar.   Interestingly Boyd's had event-driven thinking.    

The OODA loop was aimed to describe event-driven decision by a human - the human has to observe that an event happened, perform mental self orientation to analyze the meaning and implication of the event,  decide what to do, and act accordingly, have feedback loop to see whether the observation has changed.

The 4D describes a computational process, where the event is detected (not necessarily directly observed),  situation is derived (by computational means and not by mental process), and then a decision is taken (autonomic or manual) and an action is performed.   

The mapping is not 1-1:    

detect is always mapped to observe;

derive can be mapped to observe - as the detected situation is a derived event, and sometimes to orient - as it may derive a conclusion.

decide can be mapped to the combination of orient and decide in the OODA loop

do seems to be always mapped to act. 

More thoughts about the 4D and related stuff - later.  

Waters special report on CEP (sponsored by Apama)

Waters published a special report sponsored by Apama (in transition from Progress Software to Software AG),   I wanted to copy one of their figures, but they had "no copy" notice, thus I've posted some cool picture of water instead.   

The report views the Apama view on the event processing use in financial markets and include some articles contributed by the Apama guys, and some articles by financial market people.   It is interesting to note their audience polls.  They don't specify exactly who the audience is, but they probably refer to financial market  organizations.    

Some highlights:

14.9% of the organizations are using single CEP platform,  25.5% prefer to use their own solutions, and 44.7% use mix of their own code and platforms.     I guess that in other industries the percent is somewhat lower both in use of event processing and in use of COTS products. 

Risk and compliance became the major reason to use real-time analytics based on event processing, while the front office trading is second.  We see risk and compliance taking its place as a major type of event processing application with the move to continuous applications.   

Performance is again the major criterion for event processing (with the "big data" trend) and ease of development is second.   

On actions and success criteria in event processing

I am now in Durham, North Carolina, the second stop in the USA business trip (that included 4th of July dinner with friends).   A reflection on one of the DEBS 2013 presentation given by Steffen Hausmann, a PhD student supervised by Francois Bry about "complex actions".   The model built specifies an action model for event processing.  The interesting insight is that unlike traditional systems where the success is determined by the successful completion of a task, event processing systems are goal-driven, an action goal is a reaction to some situation, and is intended to satisfy some goal with respect to this situation, for example: resolve a problem.   Thus in the presented model each action has success criterion that determines whether the action satisfied the goal.    In the presentation this criterion is an event, typically emitted by a sensor, but this can be generalized to a more comprehensive model.   The idea to define success criteria for actions as part of event processing system is very useful, and in a sense closes the loop of sense and respond.   

On DEBS 2013 grand challenge

The DEBS challenge was introduced first time in DEBS 2011, since then it also became "grand". 

This year it was around football (or soccer for the USA guys) and consisted mainly on aggregations, but also detection of proximity between player and ball, using sensors.   There was a lot of interest this year, and 14 submissions.  It becomes a "community building" theme.    6 solutions arrived to the final and were invited to present at the conference, a lot of effort has been dedicated, and we saw very nice and creative solutions.  The main criteria was performance that had to be measured on a given data-set.    In the business meeting I raised some ideas to make it even more interesting, by given "unseen" data at the conference itself, and introduce last minute change in the specification.     The conference is now at its last session.  Next year - DEBS 2014 is planned to be in IIT Bombay, India.

More impressions on the last two days in DEBS 2013

The social event of DEBS 2013 was a guided tour in the Dallas Cowboys stadium, and a dinner in a nearby restaurant.  Here are some subjective highlights from the last couple of days -- I captured some isolated points, not the entire picture:

1. An interesting paper on intra-query parallelism, which in my terminology is the optimization of a single event processing agent, after filtering-in only the relevant events.  This paper came from ETH Zurich.

2. Zohar Feldman - member of our own team gave a talk about "proactive event processing in action", discussing application in the logistic area, as part of the industrial session.

3. Another interesting talk in the industrial session was delivered by Mauricio Arango from Oracle -- on mobile QOS management using EP.   Seems that Oracle is advancing its spatial event processing.

4. The second keynote -- on smart grid platform by David Wollman from NIST,  talked about platforms, standards and challenges

5.  There are two topics I'll write in follow-up posts: the grand challenge session, and a paper about complex actions.  

On DEBS 2013: First keynote speaker - Roger Barga from Microsoft

Roger Barga has been an excellent choice (of the organizers) for first keynote in DEBS 2013.

He has a wide perspective from the multiple roles he occupied over the last few years in MSFT.   

Roger mentioned Thomas Kuhn's seminal work on the structure of revolutions - saying that there are two competing forces, those who push for paradigm shift and those who resist them and try to find "good enough" way to resolve everything in the old paradigm where they feel comfortable.   The use of event-driven thinking is just a paradigm shift, this is consistent with our tutorial given in the previous day.  

Some comments I took while listening to Roger:

1. Evolution of analytics:  analytics 1.0 - descriptive analytics based on warehouses, analytics 2.0 - big data, with NOSQL and Hadoop, we are going to analytics 3.0  -"rapid analytics with business impact".    Hadoop will become a niche technology, and real-time analytics (based on event processing) will take over. 

2. Some examples: Rols Royce is giving "engine up-time" as a service instead of selling aircraft engines, with a lot of instrumentation for maintaining the engines.  Other examples in the area of telemetry.  Some companies are making huge investments on real-time analytics.

3. Today there are two types of analytics: operational analytics  based on the speed of business, but use little information to get decisions;  investigative analytics based on the speed of  data scientists, and is based on a lot of information. There is a gap between the two that need to be unified.

4.  People don't know how to make use of new technologies to find new useful applications and tend to apply new technologies to old applications -- example: the first TV area was just visual radio, until the industry learned that TV opens new opportunity.

5. Bottom line: there are velocity pressure to do real-time analytics, but it requires paradigm shift and education.  Very compatible with our conclusions.   More about DEBS 2013 - later.