StreamInsight as a cloud service

The idea of using event processing in "software as a service" mode is not new;  thanks to Marco's Blog,  I came across an article describing the StreamInsights cloud service.  The rationale behind it is that it should be used in cases that it is more cost-effective to customer relative to purchasing expensive hardware needed for processing large volumes of events.    This is part of the general trend to use the cloud for various purposes.  I assume that this will be one of the main ways that event processing will be consumed in the future. 

IFTTT - additional cool event-driven tool

I have written earlier this week about "world stream computing" as the new web paradigm, where the web becomes event-driven based on streams of events.   We can see some mobile based applications that conform with the paradigm.  I have written before about On{X}which is based on "event-action" recipes.

Another member in this family is IFTTT (acronym for: if this than that).  Like On{X}, IFTTT also uses the term recipes, it has a large set of shared recipes, which have visual icons (e.g. the Twitter Icon, the Facebook icon and more). 

Some recipes examples are:

Turn off light (on the mobile) when sun rises 

When sometimes tags my name on Instagram, send me SMS

If I check-in into an office, create LinkedIn status

As one can see the events can be events related to various social media, or other feeds (such as weather feeds),  and the actions are actually of the same type.

I have gave as an assignment for students in a seminar last semester to experiment with some tools, and experiment on people who are not IT professionals, if they can get it.  One of the student teams chose IFTTT, and succeeded to get two non IT professional people to download it, use it, and create some new  recipes.  It took them some iterations to learn it (it is not that trivial), but after an hour, they could generate recipes without problem, and noted that it is working smoothly.  

I guess that these are relatively simple tools that precede the new "active" web wave...

World Stream computing to replace the Web

I came across an interesting article by David Gelernter.  Gerlenter, a Yale Professor, who co-introduced the term "lifestreams" in the 199o-ies, proceeds in the same spirit and talks about "worldstreams" which he defines as a collection of heterogeneous real-time messaging stream  (may be thought as a collection of streams).   His prophecy is that this will be a disruptive technology for the web, browsers, search, operating systems.  We will no longer surf in sites and search the web,  but we'll browse the worldstream, and ask the worldstream to bring us what we want, in the time we want it, instead of search for it.    The event processing grand challenge, which was part of the event processing manifesto,  and talked about the "event fabric" was similar in nature, but the manifesto authors did not claim that the web will be replaced,  it will probably not happen in one step anyway.    However -- certainly interesting observation, and promises bright future...

On event model and decision model

In the article about "event model" by Luckham and Schulte, on which I have reported last week,   the term "decision model" was referenced as one of the complementary models to an event model.  This has created some discussion within the DMN LinkedIn group, the discussion was created by Paul Vincent who is one of the persons behind the DMN standard work.   I am following the DMN work, though not in an active way, and participated in the kick-off meeting in March 2011,  and also written on "decision models and event processing"  following the talk on "The Decision Model" by KPI at the same event. 

In order to view the relationship between "event model" and "decision model", we need first to understand what is a decision that a decision model is modeling 

 The DMN presentation defines decision as "A Decision is the act of applying decision logic to one or more inputs and produce one output".    This can relate to modeling a collection of business rules, which gets a request with an associated input and returns a result  ("concluded fact") or to a result of some quantitative model (such a scoring model or optimization model). 

In comparison event model is modeling event-driven logic, which is based on deriving situations or derived events based on filtering, transformation, aggregations, and pattern detection over a collection of input events.   

The question is what is the relationship between these two terms?  

we can start by looking at the relationship between business rules and event processing,  there have been long discussions about it in the past, the position I expressed was that related to a discussion with James Taylor in 2008, stated that event processing and decision management are different issues.  A decision may not be related to event, and an event processing may not trigger any decision.  However, there are also relations between them.   

In my recent presentations about event-driven applications I am using the "4 D" that coined by Jeff Adkins: 

 I found out that this is an easy way to explain both IT and business persons who are not familiar with event-based systems what it is in a nutshell.      Referring to this picture, an event model deals with the "detect" and "derive" phases.    The derived event may also by itself denote a decision (since it is obvious what to do if this situation is detected and does not require further decision),  or that the detection of this situation triggers a decision that can be modeled by a decision model (note that the situation may trigger a task, and then the decision may be embedded inside a business process model).  In the example referred to in the slide above,  the  "derive" phase detects that a traffic jam is coming, and the "decide" phased determines how to reset the traffic light policies in order to reduce the traffic jam -- in this case they are complimentary.

Another interesting point is that "decision" in the common interpretation is "request driven",  I have recently written about the distinction between "event driven" and "request driven".   One may claim that a request is an event (which is sometimes semantically doubtful), or that there is always some event that causes a person or a system to make a request (may be metaphysically true, but the event is not explicitly exposed).  However, request-driven is taking an action by request (a passive approach), while event-driven is taking an action without a specific request, due to either occurrence of event or detection of situation (an active approach).    An event can take 

Bottom line:  decision models and event model are complementary models.   I'll discuss similarities and differences in the content of what is modeled in a later phase. 

Event model - what comes first - the event data model or the event driven logic model?

As promised, I'll write some issues related to event modeling, which is my current focus of interest.

The current thinking on event processing and actually on some related areas (such as BRMS), and also reflected in Luckham and Schulte's article on event models is that the first step is to build the data model, and then model the logic on the basis of the data model.     According to this methodology there is a need to model the used data and events, sometimes also modeling "business objects" that maybe created from the original data or events.   When modelling the logic it always relies on the existing events and data already modeled.  This is a "bottom up" approach.    

 A major issue with this methodology is that it moves the modeling immediately to the IT domain, since business users can relate better to goals, processes, and business logic then to data models.   Keeping the modeling in the level of business require to reverse the process,  model the business logic first, and leave the data and events as "place holders" which need to be completed later in the process.  

A top down modeling is also reflected in that article when the phase of "identify the complex events" precedes the phase of "identify base events".   I think that in the business user's terminology it is better to talk about situations that need to be reacted upon then on "complex events" that is more technical (and somewhat ambiguous) term.   Of course, modeling is an iterative process, since when getting back to the base events or data, a gap (e.g. event that is not being observed) might be discovered.   Thus it seems that the methodology should be:  start top-down (goal oriented) and tune up bottom-up.

More on event modeling issues -- later. 

Event processing with Storm-Cassandra from health market science

I came across a recent Slideshare presentation entitled "Complex Event Processing W/Cassandra". prepared by Brian O'neill and Taylor Goetz, from Healthcare Market Science.  It describes a project integrating Cassandra and Storm.   This presentation analyzes previous failures using Hadoop and C* with  aspectj.

Then it explains the architecture of the solution using Storm for the event processing part.   

Interesting presentation.  It would have been even more interesting if the presentation talked more about the actual application  (there is one slide explaining in very general terms what their products are).  

It seems that a lot is going on in the open source space.  

Event model - the next frontier

I am writing this time from Falls Church, Virginia.  Where I arrived yesterday for a business trip in the Washington DC area, and then in Toronto.   So far I was not affected by the weather, but we'll see what happens later in the week.     

The slides above is an old slide which I used seven years ago to show the relationships of events and the rest of the universe.   I was reminded on that when reading a new article by David Luckham and Roy Schulte entitled "Why companies should develop event models?".    In this article they explain the motivation and provide a high level methodology to build event model.     Following what I recently written about "event oriented thinking",  it seems that the major obstacle to adoption of event-driven systems is the ability of people to think in event-driven way and not in the traditional request-driven way.   Being able to create event model and relate it to enterprise computing modeling is an essential step.    This is the direction I am working on these days (very much in the direction that Luckham and Schulte indicate), and will surely write more about it in the sequel.