On{X} Recipes - cool event based Android application

Thanks to Guy Sharon I've learned about cool Andorid application developed by Microsoft Israel called On{X}.  It enables to specify and use an event- condition-action recipes (the name sounds tastier than rules).

Examples of recipes are: 

Ring on the third call from the same person when my phone is silent 

Set mode to silent between 11:00PM and 7:00am after the phone has not been unlocked for 2 hours

Text my wife "my phone is dying" when the battery goes below 15%

All of these look to me as event patterns.  Looking at the On{X} Blog I learned that new features include - dynamic regions, noise detection and more.  There is a collection of recipes and one can program more using JavaScript 

A very interesting event processing application  -- I may buy smartphone one of these days :-) 

More on request driven vs. event driven

In the table above (taken from a recent presentation I am working on) I have summarized some of the main differences between request driven thinking and event driven thinking.   It is interesting to note that many of the activities we are doing in life are event driven,  however, we are programmed to think that computer should be approached in request driven way, and I have noticed that even if the application itself is event driven by nature, people will tend to convert it to request driven.  Event driven action is being activated not due to explicit request but since an event has occurred, or a derived event was concluded.  This may happen in unknown time and unknown frequency.  Furthermore, a request getting into a system should always entail response (which can be error message),  an event getting into the system may be ignored, since it is out of context, just increment internal state, or close the circle of detecting derived event which can be either internal to the system, or trigger external action or notification,  Note that only the last case has visible response to the outside.    One of the challenges is to educate people to think in event driven way rather than request driven.  I'll write more on event driven thinking in the sequel. 

Container fleet management with event processing

An interesting application of event processing technology is reported by Jack Vaughan on container fleet management done in  Orient Overseas Container Lines (OOCL), Hong-Kong based company.

It tracks various events such as monitoring the load time to avoid penalties.   You can read more in the article itself.  

On event oriented thinking

The title of this Blog is "event processing thinking", and much of the posts are my own thoughts about event processing, but today I would like to write about another topic: event oriented thinking.  

An interesting observation from recent discussions is that  in daily life we can very easily think in an event-driven way,  when the phone is ringing we are either answering it or ignoring it, when we hear about traffic jam - we try to find another way, and so on...      However - when it comes to thinking about computerized systems,  many people are programmed to think in a request-driven way, which means:  a person sends a request and the computer responses.   Thus people are trying to use what they know even if they have scenarios that are event-driven.  A very typical line of thinking is:  event is a data, we should insert it in a database, and then ask query that reflects the situation that we wish to detect.   The question is -- when are we going to ask this query.  We can ask it on any update -- but if the event is just one component in a pattern we wish to detect (example: we wish to detect when a customer sent three requests within a single day -  assuming that most customers don't send more than one request, then we can have 99.99% of the queries redundant),  we can ask the query periodically - but then again, we can both ask redundant queries, and moreover, not able to react on time, since the granularity of our timing is wrong.   There is also a notion of continuous query -- but this is not really a query, since it is not a result of a specific request.  

The nature of event-driven scenarios are: we don't know when they are going to happen, we even don't know whether they are going to happen,  but when they happen we want to do something - sometimes very fast (e.g. earthquake detection).   Furthermore, the situation we would like to detect can be realized in a pattern of many events,  and each individual event may trigger reaction, just increment some state, or even be ignored or filtered out.

The fact that people are trying to model and implement event-driven scenarios using the traditional request response is a thinking mismatch, and creates added complexity.  I'll follow up with an example in one of the next posts.  More - later.

On fifty years of databases

This drawing is taken from the ACM SIGMOD Blog post by Thomas Haigh entitled 
"Fifty years of databases".   It tells the story of  IDS (Integrated Data Store) as the first database developed 50 years ago in GE.  IDS was designed by Charlie Bachman, who received Turing award for his pioneering work on databases, and as Haigh remarks  - Bachman was the first Turing award winner who did not have PhD, and actually spent his life in industry and not in academia.  It is worth remarking that the two persons who followed Bachman by receiving Turing awards in the database area, Tedd Codd  and Jim Gray were also from industry, in fact, the academic database community did not have until today any Turing award winner.  Bachman's Turing award talk "programmer as navigator" was very insightful.  Bachman compared himself to Copernicus who said that the earth is revolving around the sun and not vice-versa, and said that the computing world will revolve around data - where programming will be side effects of operations associated with data.   We are not quite there, but for me it was an inspiring goal.
IDS is far from the current databases we have today, but it laid the principles, and started from pure engineering position, theory came later.     I am trying to compare the database area development to those of event processing, and think that in many respects we are still were databases have been 40 years ago,  so the challenge is to advance it further...  more on that -later.

On decision latency

James Taylor wrote in his Blog an article on decision latency, the illustration above is taken from his article. 

The latency for getting a decision consists of three time intervals:

The Capture latency is the time taken from the event occurrence in the universe until it is detected (captured) by the processing system

The Analysis latency is the time taken from the time that the event is detected by the system, until the time it is  processed and creates the derived event/data that triggers an action, and the Decision latency has two parts - which I think should be distinct,  the time to make the decision how to react, and the time it takes to react.

These four type of latency are actually orthogonal.  For the Capture latency what is required is an infrastructure of instrumentation, sensors and communication; for the Analysis latency -- the event processing system latency, and possibly additional analysis required by querying historical databases,  for the decision latency --  the decision tools (either a decision management system or optimization system), and for the action latency -- the speed in which it can be applied. 

Why the decision latency is important? -- here we are getting back to the real-time issue.  In some cases the time is critical for decisions,  an example that we saw a few weeks ago is in the Israeli anti-rockets systems

known as "Iron Dome" (Kipat Barzel).  The system detects that a rocket was launched, and then it needs to analyze its course and determine whether it is going to hit a populated area, in order to decide whether it is cost-effective to send a missile to hit the rocket (this decision is needed since a missile is very expensive), and if yes,  determine when and where the missile will intercept the rocket and launch the missile at the right time to the right direction.  Latency in all the four phases is critical., and the success of this system saved many lives.    

Decision latency is not only related to defense systems,  it exists in different other areas - healthcare is certainly one of them, but also business.  The time scale does not have to be seconds,   decision latency can span minutes or hours,  

We can look at the value of decision as function of its latency,  and can see that there are soft real-time cases which the value of decision goes gradually to zero, Firm real-time where it goes to zero at a certain deadline.

Hard essential in which it goes to a certain penalty (such as missing SLA), and hard critical where it is going to a minus infinity (a disaster case), the Iron Dome case is of that type.  

Of course, there are no miracles, and the cost of a system grows when bounded latency need to be guaranteed (for example cost of extremely robust communication).  For each type of event-based system there is a need to calculate the trade-off between the value of latency and the cost. 

Launching the online magazine: Real-time Business Insights: Event Processing in Practice

There are two events today -- the 15th birthday of my daughter Daphna (in the picture taken in our recent family vacation in Malta)

Daphna's birthday will help me remember also the other event happening today -  launch of the online magazine.  I have written about the idea of the online magazine before,  a few months ago - took some time to accomplish - assemble and editorial board, solicit material from authors and find a publisher who believes in the idea to take care of the logistics.   

After some deliberations - the name chosen is shown on the Logo below.  The idea is to start with several pilots issues, the first of them is being launched today.  The magazine consists of several sections:

Business Strategies presents the customer's perspective on how other enterprises gained value from processing streaming data and events in a variety of use patterns.

Tools and Tactics presents the practitioner's perspective with best practices and lessons learned from those who develop event-based solutions.

Frontline presents the community perspective with provocative and thought provoking opinions.

Industry Insights provides in-depth education about the use of real-time business insights in a particular industry.

On the Horizon provides the researcher's perspective on what's coming next in

applied research projects and new technologies.

 EP News keeps readers abreast of the latest industry developments

Each of these section has a section editor, I volunteered to serve as editor-in-chief and section editor of the blogging section ("On the Horizon") - to set a personal example, I have also written the first one, expressing an opinion which will come at no surprise to this Blog's readers.  

The online magazine can be downloaded from here:   (registration is required by the publisher), 

You are encouraged to download, react (the magazine's website links to the EPTS LinkedIn group), distribute to anybody who might be interested, and contribute to any of the magazine's section.

Enjoy and react!