58 sensor applications

I came across a site that lists "top 50 sensor applications for the smarter world".  It actually list 58 applications partitioned to the following areas::  smart cities, smart environment, smart water, smart metering, security & emergencies, retail, logistics, industrial control, smart agriculture, smart animal farming, domestic & home automation, and eHealth.  

I worth digging into the different areas to check the potential applications, and the role of event processing in each of them. 

Gartner's recent predictions about business intelligence and analytics

A recent set of predictions by Gartner, state that Business Intelligence and Analytics will remain top focus for CIOs through 2017. 

It mentions two interesting observations:   One is that the confusion in the market about the term "big data" and its tangible results constrain the spending and limit the growth of BI and analytics software.   The second observation is that By 2017, more than 50 percent of analytics implementations will make use of event data streams generated from instrumented instrumented machines, applications and/or individuals.  

This is consistent with the Gartner's term -  "two tier analytics",  where event processing is the second tier, after historical data analytics.  While the need to consolidate analytics and event processing is becoming more pervasive, the utilization barrier and the need to battle complexity is still a common denominator. 

On continuous compliance

The Waters special report sponsored by Apama that was published recently,  ranked "risk and compliance" as the number one application of event processing in financial institutions.   Today TIBCO also published in its Blog under the "event processing" category its view on compliance - entitled: "The Y in comply".   It gives the view that people comply better when they understand the rationale of compliance, the example given is compliance with regulation in the food industry that employee has to wash hands before putting on gloves.  

I agree that people comply better when they think that the regulation makes sense,  however, my own subjective impression is that in the current culture, compliance became goal of its own whose sole reason as somebody described it "to get the regulator off my back".   Thus employees are required to do things that they don't see any reasonable explanation why they should do it, and IT systems check that they are doing it. Here, event processing is used  since in various cases auditing is event-driven, it is time sensitive an often involves calculation in time windows, and in some cases online auditing on the fly is required. The business value sometimes is just pleasing regulators, but this is often a great motivation for managements to take it very seriously...   Anyway, technologies are used for strange reasons sometimes...

Proactive event processing for intelligent transportation system

I came across a new publication whose citation is: " Yongheng Wang, "A Proactive Complex Event Processing Method for Intelligent Transportation Systems," Lecture Notes on Information Theory, Vol.1, No.3, pp. 109-113, Sept. 2013. doi: 10.12720/lnit.1.3.109-113".   This paper is a follow-up to our work on proactive event-driven computing,  and applies events coming from the Internet of Things towards intelligent transportation system, proactively mitigating traffic congestion.  This work originates in China, who made Internet of Things as its flagship project.   We have looked at similar problem  as one of our use cases for EU project proposal (that did not win the lottery).

  

Event processing in gaming: SAP and Bigpoint

Jeff Wooton from SAP posted on the SAP event processing blog a video clip showing a project done with Bigpoint, which is a gaming company, to get events from players in real-time and make offers to players based on their progress in the game and prediction of patterns of their behavior.   The solution is based on Hana, SAP in-memory technology, and I guess on their event processing technology also, though the video does not give much details about the actual implementation. The goal is to maximize profit by personalize sells to game players.   Event processing has various applications in game playing, as game has event-driven behavior nature, and also in dynamic personalization, which is applicable to other areas as well.

IFTTT - additional cool event-driven tool

I have written earlier this week about "world stream computing" as the new web paradigm, where the web becomes event-driven based on streams of events.   We can see some mobile based applications that conform with the paradigm.  I have written before about On{X}which is based on "event-action" recipes.

Another member in this family is IFTTT (acronym for: if this than that).  Like On{X}, IFTTT also uses the term recipes, it has a large set of shared recipes, which have visual icons (e.g. the Twitter Icon, the Facebook icon and more). 

Some recipes examples are:

Turn off light (on the mobile) when sun rises 

When sometimes tags my name on Instagram, send me SMS

If I check-in into an office, create LinkedIn status

As one can see the events can be events related to various social media, or other feeds (such as weather feeds),  and the actions are actually of the same type.

I have gave as an assignment for students in a seminar last semester to experiment with some tools, and experiment on people who are not IT professionals, if they can get it.  One of the student teams chose IFTTT, and succeeded to get two non IT professional people to download it, use it, and create some new  recipes.  It took them some iterations to learn it (it is not that trivial), but after an hour, they could generate recipes without problem, and noted that it is working smoothly.  

I guess that these are relatively simple tools that precede the new "active" web wave...

On airport events

Tomorrow I am planned to visit three airports,  starting from Ben-Gurion airport near Tel-Aviv to takeoff from Israel, moving through Frankfurt airport in Germany for connection (hopefully no serious delays, it seems to be snowing in Frankfurt),  and ending in the Dulles airport near Washington DC.   I have reported in this Blog in the past on several flights related events, most recently - getting back from DEBS 2012 in Berlin 49 hours later than planned.   Airports are full of events,  and it was interesting to read Pedro Garcia blogging about  event processing as part of airport management system.    In an airport there are many events related to flights, luggage, strikes, weather and more.    The events that Pedro relates are both event that happened and predicted events (flight delay can be predicted).    While some of the delays are force major, others are certainly created by people (I have plenty of stories about human inflicted events).   Improvement of flight management, as well as improvement of luggage handling systems, can be very helpful.

It is also interesting to note that the system is based on the architecture we presented in the EPIA book, which is explicitly mentioned.  Always good to see that our work is useful to somebody :-). 

Hoping for uneventful flights (have several more flights during the coming week). 

On{X} Recipes - cool event based Android application

Thanks to Guy Sharon I've learned about cool Andorid application developed by Microsoft Israel called On{X}.  It enables to specify and use an event- condition-action recipes (the name sounds tastier than rules).

Examples of recipes are: 

Ring on the third call from the same person when my phone is silent 

Set mode to silent between 11:00PM and 7:00am after the phone has not been unlocked for 2 hours

Text my wife "my phone is dying" when the battery goes below 15%

All of these look to me as event patterns.  Looking at the On{X} Blog I learned that new features include - dynamic regions, noise detection and more.  There is a collection of recipes and one can program more using JavaScript 

A very interesting event processing application  -- I may buy smartphone one of these days :-) 

On the Internet of EVERYTHING

Cisco came out recently with the concept of  "The Internet of EVERYTHING".

While the "Internet of Things" deal with connecting anything to the Internet, the Internet of EVERYTHING deal with the things and the semantic connection among things that can make the world actionable in real-time.  A simple example is the car theft example. 

A car is connected to the Internet through its GPS sensor that reports its location, it has also semantic relations to a list of eligible drivers who are permitted to drive in this car, each of these persons is also connected to the Internet using his or her mobile phone, thus the Internet knows the person's location  (disregard the privacy issue for this scenario!),  so if the car is moving (inferred from the GPS change in location), and all the eligible drivers are not in the car - it means that the car is stolen, and it can then report to the police and have them track its location.  

This scenario is based on - things,  contexts of things, and processing events about things..   It is actually quite straightforward from technology point of view.  I wonder if the "Internet of EVERYTHING" will survive the buzzword test of time... 

Adding voice to event processing

Richard Seroter reports on a demo he constructed using the cloud version of Microsoft Streaminsight and Twillio services to get situation detected by event processing and then send a voice message as a notification,  he also outlines the code behind this demo (a video clip with the demo itself would be cool addition). 

In this case the voice is being used as event consumer to send notifications,  it will also be interesting, but of course more challenging to have voice as producer for events,  actually there are some works done on this area and I'll try to get more details and report on it. 

More on Internet of Things and event processing for personal systems

This illustration is taken from a recent Blog posting by Phil Windley, the CTO of Kynetx, whom I met in DEBS'11.   Phil writes about IoT as a source of personal event processing, and about "personal event bus", in which any consumer can subscribe to events and process them individually.


I have provided some background about IoT recently and about its potential to be game changer;  the personal systems is an interesting aspect.   Event processing models and systems of the first generation have been developed in mind of business oriented systems. However, I have stated before that I see the big potential of event processing outside the traditional corporate IT, and provided robotics and biological computing as some areas of interest.  Consumer applications is definitely an area with a lot of potential, however the challenges to do it is beyond collecting and subscribing to events, it is the ability to compose event processing application by consumers, which requires progress in the abstraction level of composing such applications.  This is definitely and avenue to pursue. 

Event processing = big data + real time ?

BIG DATA is a hot phrase, one can also be labelled as "BIG DATA NERD" by purchasing the shirt shown above.   The term BIG DATA refers to the explosion of data in the universe, especially methods to store and process huge amount of data.   


Paul Vincent poses the question:  CEP = real-time bid-data?     Actually he doesn't answer this question on his blog posting, but the spirit of the postings makes the impression that the "=" relations is evaluated to "True".    


The answer is that it is not really an equality, but more like an intersecting concepts.   


Event processing of course does neither necessarily works on big data nor it necessarily has to satisfy real-time constraints.


On the other hand,  real-time big data, is not necessarily event-driven and process events at all, as big data can be fairly static.    As an example for real-time big data, I'll return to an invited talk in DEBS 2011 on Watson, in which there has been a live demonstration of  the Jeopardy! game with the Watson computerized system, you can see in the picture that Paul Vincent is playing (very skilfully) the host.  

Watson is a "big data" crunching application, it has very strict real-time constraints, so it certainly can qualify as "real-time big-data", alas, it has nothing to do with event processing, in fact the data exists prior to the game  and no additional data is added during the game.


While the equality does not work,  there is an obvious relation;   in some cases there are substantial number of data inputs, which much of it can be reduced, or filtered out.   Event processing can be used to process streaming data on the fly, and not follow the paradigm of store now process later.   This is important especially one the processing is required in real-time.


More on event processing and big data -- later

On terminology again

While the EPTS glossary team is about to produce additional version of the glossary,  I received by Email a question about terminology from a student investigating this area, whether "event driven architecture" and "event processing"  (both titles of books) are indeed the same thing  


The answer is -- not really, but the difference is simple:   


Event Driven Architecture is an architecture in which there are event producers, event processors, and event consumers, with the following principles:  1). All players are decoupled;  2). All the communication between players is that they send events to one another:  3). All communication between players is asynchronous. 


Event Processing is a type of processing that takes as input one or more events and produce as output one or more event applying functions such as: filtering, aggregation, transformation and pattern matching.   


The concepts are related in the sense that it is common to implement event processing on top of event driven architecture, but it is not tightly coupled:
Event Driven Architecture can implement  pub/sub systems that are doing routing only and do not do any processing of events;
Event Processing can be be implemented in synchronous mode and not on top of  EDA.

On Watson and Event processing

This is a picture taken in the DEBS conference last week, we had an invited talk by Eddie Epstein from the Watson team, and he ran a round of Jeopardy!, two of the conference participants against Watson, here you can see that Watson has -800 point (click on the picture to see it clearly), but it recovered and won.   We invited the talk on Watson, not because it somehow related to event-based systems, but because the conference took place in the Yorktown auditorium, where the famous Jeopardy! game in which Watson succeeded to beat two of the all time champions was recorded.


Today I've found a Blog posting by Shalin Shah from Vitria, with the promising title:  

IBM’s Watson: What Does Complex-Event Processing Mean For Customer Experience Management?

So I tried to understand what the author thinks is the relationship between the  two,  the answer according to the posting - both of them can be used for operational intelligence.

Indeed, there are now efforts within IBM Research, to determine what are the next steps, since in essence Watson is "deep question answering machine" there are some areas that seem to be killer applications of this technology, among them are: medical diagnosis and helpdesk/contact center in which agents need to answer questions in a lot of areas. There are some others as well.   

From technology point of view, Watson works in a different paradigm relative to event processing.  It is not event-driven, but is based on a knowledge stored in books, encyclopedias, and other sources.  What it does in real-time is - question understanding and question answering using statistical reasoning, and massive computational power.   

The interesting question is what can be a synergy between question answering machine and event processing,  here we can think of two sides:   an event processing system is being assisted in Watson-like system in order to determine contextual information that can be used for evaluating assertions, or classify events into context instances.   On the other hand a question answered can trigger event.  Or the question answering system can be monitored by an event processing system.     One can also think about real-time update of Watson's knowledge-base as a result of event, which is not the way Watson currently works.    I think that there are various more synergies between the two types of systems.

More - later.

Smart cars -- towards proactive elimination of car accidents

Thanks to a posting of Rainer von Ammon in the CEP forum,  a "killer" application of proactive computing,  is the MIT smart cars that predict human behavior of other drivers and intended to eliminate accidents.  This is intended for the interim period where some of the car will be autonomic and some manually driven.  The rate of success is high, still not perfect.   Actually this is the opposite of a killer application - it is a life saver. 
I guess that in the future, all cars will be driven by computers and this will be able to eliminate the accidents (or minimize them),  since human behavior cannot be totally predictable,  More on ability to predict events - later.   

Processing events for discoveries in physics

The STAR experience made some discoveries about the structure of the proton using real-time processing of events.   The notion of events and their processing has been used in physics for a long time.   It seems that such processing require enormous computing power.   Processing of events is done in many domains, besides business applications. 

Reviewing project for the European Commission

Dortmund, Germany.     Morning -- packing to start the way home.
In the last couple of days I have participated as a reviewer in two reviews of  EU projects which are centered around event processing -- Play (in Brussels) and Pronto (here in the fire fighters instruction center).   Play is in his first year,  building infrastructure for  "events marketplace",  Pronto in its third (and last) year, deals with events that arrive from multi-media sources, you can read some details about them on their websites.   Getting more event processing centered projects funded by government agencies is an indication of the growing importance of this area.    Got some interesting observations out of these reviews, which I'll write about later.

The human body as event producer

I realized that I have not written anything in this blog, in the last week, well - there are weeks in which one is in a passive mode, anyway, I am at home, ready  to go to the office (Sunday is a working day in Israel, our weekend is Friday-Saturday), but wait for the technician from the phone company, all our phone lines are dead since Thursday afternoon.   Since each member of the family has cellphone, it is not a big damage, the most notable damage is that the wireless internet at home is linked to the phone (ADSL),  however, we have a second home LAN (wired) which is based on the cable TV, since none of the internet connections are reliable, they also serve as a backup for each other.   The phone company was a government owned company that was privatized, but retained a strong employee union as a legacy, and as such, its service is not that good, so we'll see when the technician will bother to come.


Anyway -- the picture above is of stream of events going out directly from the human brain, it is taken from a recent article in Scientific American.   There is a lot of recent work about brain-computer interface, the brain itself is, of course, an event processor, it receives events through the senses,  makes some decisions, and acting with some actuators.  There are also a lot of events being created in the human body  that has benefit to be processed elsewhere, either due to fact that we want to have "extension" to the brain to help sort out the events and detect situation that the human brain may miss, or that the human body is just one source, and there are other sources for events that this particular human is unaware of.     


I see a lot of future in taking what was done in event processing and utilize it outside enterprise computing, and I'll investigate this theme more in the next few blog postings.