Event processing with Storm-Cassandra from health market science

I came across a recent Slideshare presentation entitled "Complex Event Processing W/Cassandra". prepared by Brian O'neill and Taylor Goetz, from Healthcare Market Science.  It describes a project integrating Cassandra and Storm.   This presentation analyzes previous failures using Hadoop and C* with  aspectj.

Then it explains the architecture of the solution using Storm for the event processing part.   

Interesting presentation.  It would have been even more interesting if the presentation talked more about the actual application  (there is one slide explaining in very general terms what their products are).  

It seems that a lot is going on in the open source space.  

Event model - the next frontier

I am writing this time from Falls Church, Virginia.  Where I arrived yesterday for a business trip in the Washington DC area, and then in Toronto.   So far I was not affected by the weather, but we'll see what happens later in the week.     

The slides above is an old slide which I used seven years ago to show the relationships of events and the rest of the universe.   I was reminded on that when reading a new article by David Luckham and Roy Schulte entitled "Why companies should develop event models?".    In this article they explain the motivation and provide a high level methodology to build event model.     Following what I recently written about "event oriented thinking",  it seems that the major obstacle to adoption of event-driven systems is the ability of people to think in event-driven way and not in the traditional request-driven way.   Being able to create event model and relate it to enterprise computing modeling is an essential step.    This is the direction I am working on these days (very much in the direction that Luckham and Schulte indicate), and will surely write more about it in the sequel. 

On airport events

Tomorrow I am planned to visit three airports,  starting from Ben-Gurion airport near Tel-Aviv to takeoff from Israel, moving through Frankfurt airport in Germany for connection (hopefully no serious delays, it seems to be snowing in Frankfurt),  and ending in the Dulles airport near Washington DC.   I have reported in this Blog in the past on several flights related events, most recently - getting back from DEBS 2012 in Berlin 49 hours later than planned.   Airports are full of events,  and it was interesting to read Pedro Garcia blogging about  event processing as part of airport management system.    In an airport there are many events related to flights, luggage, strikes, weather and more.    The events that Pedro relates are both event that happened and predicted events (flight delay can be predicted).    While some of the delays are force major, others are certainly created by people (I have plenty of stories about human inflicted events).   Improvement of flight management, as well as improvement of luggage handling systems, can be very helpful.

It is also interesting to note that the system is based on the architecture we presented in the EPIA book, which is explicitly mentioned.  Always good to see that our work is useful to somebody :-). 

Hoping for uneventful flights (have several more flights during the coming week). 

Grok by Numenta - real-time pattern discovery

Thanks to Jeff Adkins,  I have looked at the website of a company called "Numenta" which has a seemingly interesting product called Grok.  As a science fiction fan, I like words from science fiction books that made their way into the language, and Heinlien's word from the immortal book "stranger in a strange land" is one of them.  

The Grok product by  Numenta,  illustrated by this figure -

is described as a tool that discovers in real-time patterns in events (data streams) and generate predictions and anomalies detection. The technology behind it is described to emulate the human brain and belongs to the neural nets family.  There is a white paper on the website explaining it.  

The site describes the types of created patterns as: temporal, spatial, and spatiotemporal.  However, their use of the term "spatial" is non conventional in the sense, that it does not have any necessary relationship to location, but is defined as "relationships between things that happen at the same time", which in the examples relate to relations between attributes of the same event (e.g. the relationship between age, gender and income to loan amount).  Calling this relation "spatial pattern" is kind of confusing to me.

Other than that -- seems interesting, I will be curious to get more information about real-life experience of this technology. 

DEBS 2013 grand challenge was announced

DEBS 2013 grand challenge was published today.  This is one of the new items we introduced in DEBS 2011 which continues to the next generations. The idea is to publish an event processing application, view various implementation and evaluate based on announced criteria.

This time the application is around football (soccer in American English), where the input events come from sensors attached to players' shoes and hands and to the ball, and some referee events.   The events are spatiotemporal by nature. 

The goal is to calculate various statistics about players performance, ball's location etc.

Criteria -- application's performance (latency,  throughput),  while keeping correctness. 

I hope that various teams will answer this challenge and will provide interesting solutions. 

Using event processing to make "big data" becoming "fast data" by Alex Alves

As part of the first issue of the online magazine "Real-time business insights", Alex Alves wrote an article on the use of event processing in big data,  recently Alex remarked on this article in his blog,  saying that while the common big data platforms are batch oriented,  turning "big data" into "fast data" is done by combining event processing with big data technologies.     Stay tuned for the second issue of the online magazine,  now in preparation. 

On When vs. Where

Thanks to Harold Ship, I came across a post by Bill Lee entitled "Investing in 2013: It's about time not location".  Lee claims that while there have been much traction around location based solutions,  the location by itself has less value that the time of occurrence of past or future events.  In the figure above, it might be more interesting WHEN Achilles is going to bypass the tortoise than where it is going to happen.  Saying that a traffic congestion is going to occur at a certain location might be meaningless without prediction about the timing of the traffic congestion.   It is more valuable to know when the delayed aircraft is going to land than just to know where it is now, it is more valuable to know when the technician is expected to arrive than just to know where he is now.  I don't really care where the guy who delivers my pizza is now, but I definitely want to know when he is going to arrive.   Lee claims that in the USA alone businesses lose $90 Billion annually due to people running late, whether it is the employee, or the technician for whom the employee waits at home.   The claim is that time-based services is a good topic for 2013 investors to pursue, whether location is involved (spatiotemporal capabilities) or not.   As the timing of these time-based services is associated with current or predicted events,  event processing is a key for such services.  I'll write more on time-based services later.