On reaction to events

I noticed that the number of my linked-in contacts has grown again to a nice number 888,  my wife said that if I'll hit 999 she'll join linked-in and be my 1000 connection, so have some way to go.  



Anyway - one interesting insight from my recent visit in the USA is the way to react to event, or why proactive behavior needs to be smart.  I was driving in a road and there has been a sign notifying: traffic jam in a certain bridge, it is advised to use alternative roads, there was still around 30 minutes to drive to that bridge, and a couple of alternative roads. 




 My own calculation was that most of the traffic heading to that direction will use alternative roads, and that there is enough time for the traffic jam to be cleared, assuming that there will be very little additional traffic,  this of course was a gamble, but since I was not in a hurry to catch a flight, I decided to take this gamble, ignored the advice and drove straight to the bridge with the traffic jam, and surprise...surprise... the bridge was totally clear without any delay, indeed the traffic jam was cleared and there has been very little traffic heading in that direction.   Of course, if many other drivers would have taken the same strategy it would not have worked (like the prisoner's dilemma in game theory).     The lesson it that notifying about event may not yield the best result, maybe alternative road became jammed.  In this case the global optimization was to direct certain percentage of the traffic to each alternative route and leave certain percentage directed to the original route.   If we are looking from the point of view of individual driver who does not care about the others, this driver still needs to take into account the other's reaction in order to determine strategy.   I guess that these kind of considerations are also getting into trading decisions.  

More on the OMG event processing symposium in the capital market area

Back home,  looking at some notes I have taken in the OMG EP meeting in NYC last week.  Here are some of these notes:

• The host was Credit Suisse, so we had an opening talk of Eric Newcomer, who told us about Credit Suisse, and also told us that his title was changed from CTO to Chief Architect, since the problem is not evaluation of technology, but coping with the complexity of the systems.  

•  Roy Schulte (Gartner) gave the first keynote talk.   He set the tone of other speakers as classifying event processing as analytics (which is the current hot buzzword),  he said (this is a slight variation about his previous classification) that event processing have three levels: the simple event processing, which we always done, the BPM + state machine which have some event-driven functionality, and the analytics event processing - AKA CEP or ESP, i.e. providing aggregations and pattern matching.  On the financial market front he said that while the original EP applications have been in the front office, it is now getting to the middle and back office in various application areas.   He also sees that the future of inter-systems integrations will be in using events.   On EP as part of analytics he also talked on the BI classification -- descriptive (BAM systems),  predictive (pattern matching is a way to define predictive) and prescriptive (which is still mostly futuristic).    There are also three ways to deliver event processing:  Basic engine,  EP platform (which includes tools for management, debug etc..) and packaged applications  (this classification is not new).

• Thomas Sulzbacher, CEO of Starview, talked about his product. He started his talk by saying that he took a flight from SFO to NYC in order to arrive into the conference, and in order to execute the flight, there were 12M  event-driven decisions required, most of them automatic. 

• David Parker, from Sybase, An SAP company, as he repeatedly reminded us, also classified event processing as part of the analytics solutions of Sybase.  The interesting insight he provided (from Sybase internal sources) is how they view the distribution of the EP market:   42% Finance Sector, 14% government, 11% services, 8% telecommunication 8% healthcare and life sciences, 6% production, 4% retail, 3% education, 2% energy and utilities.     It is interesting to see if other vendors have different perspectives, especially what analysts have to say, since no analyst has produced such a detailed distribution so far.

• Colin Clark, whom I met first time, talked and demoed his cloud event processing solution, that does indexing, clustering. classification, summarization and anomaly detection.  He came along with his business partners that made a draw among the participants that provided them business cards,  and gave some gifts -- I think it was iPad.   

These talks were all before lunch - after lunch I did not take notes anymore;  there were two talks - one by John Bates, interesting as always, given us concentration of incidents in the financial markets area, among other things, and the last talk (before mine) was Matt Meinel from Informatica (came from the acquisition of  29West).

  

Proactive computing -- keynote talk in the OMG event processing CoP conference

Hello from Philadelphia again.  Today I spent the day in NYC (got by train and returned to Philly in the evening). 
I have attended the OMG event processing CoP conference for financial markets that was held in Madison Avenue in NYC.    I'll write about the conference soon -- but for now, just a short posting about my own presentation;  I have given a keynote address (actually in the program it has been written as luminary keynote, not sure I am a luminary, but it seems that Brenda Michelson who organized the conference likes to use this term).  Anyway -- I have given in the first time a public talk about our activities in the proactive event-driven computing area,  talking also about the "first of a kind" program of IBM Research which enables to do research work in collaboration with end users.    I got several insights from the audience about various applications that I have not thought about for proactive computing, and will follow-up with some of them -- this is one of the directions to which the event processing area will evolve, and it seems to be an exciting direction to pursue, more about it - later.

Visiting Temple University - 20 years after graduation

I have finished my PhD studies 20 years ago in Temple University in Philadelphia, and after many years I have visited my old school today, and gave a talk in the CIS department seminar.  It was interesting to meet again those faculty members who are there, and some new ones.  I was especially happy to meet my advisor, Giorgio Ingargiola,  who is about to retire at the end of this year.  Giorgio has been a great mentor on clear thinking, and  a major influencer on my own development, here is Giorgio's current picture.

I have some follow-ups, especially on the software engineering of event-based systems which I may have found a collaborator.    Giving a seminar I got some interesting questions, one of them about the gap between events in computerized systems vs. events in the physical world -- which can be noisy and imprecise.   I gave detailed answer about various cases where current systems have semantic gaps (this topic is discussed in chapter 11 of the EPIA book).      Have to get up early tomorrow to take the train to NYC, for the Event Processing capital markets meeting.     

DEBS 2011 calls are finalized

DEBS 2011 calls have been now finalized and can be found on the DEBS 2011 website.  The calls are for the traditional tracks -- research track, industrial track, tutorials, demos, PhD workshop (still TBD).
The new tracks in DEBS 2011 are:

• DEBS challenge:   this will consist of an event processing application with a call for demonstration of its implementation in various approaches; this is targeted to commercial and non commercial implementations, the non commercial implementations will be able to participate in a competition where the major criterion will be ease of use.
• Gong show:  participants will have 5 minutes to present new ideas - both research idea, and interesting kind of usage in event-based systems and technologies.  The audience will vote the best idea.
• The industrial track will have an "experience reports" sub-track, that will not require submission of papers, but abstracts only. 

Note that there are four committed keynote speakers.     I'll follow the progress of DEBS 2011 in this blog. 

New review for the EPIA book

It is holiday again, and tomorrow I am leaving for a short (2 days) vacation here in Israel with my family, and then in Sunday night, travelling to the USA for a short (4 days) business trip, where the highlight will be participation in the event processing - capital markets conference of OMG EP CoP;  here is the conference's program. I have been travelling too much recently, hope that after this trip I'll have a break in travelling.


Today I've noticed a new review of the EPIA book posted by Tushar Jain, a person I have not been familiar with so far.   Good to see that people like the book.   The reviewer is right that the conceptual model we described in the book still need to obtain acceptance like UML and BPMN,  well -- the next step is to try and work on standard proposal for event processing modeling language, and I'll take advantage of my coming trip to USA to try and kickoff this activity.
One comment -- I don't see the other books mentioned as competitive.  The book of Mani Chandy and Roy Schulte is a business oriented book, and our book is a technical oriented book, so the intersection is fairly limited, furthermore, we mention in the book that we don't deal thoroughly with the business perspective, and recommend Chandy and Schulte's book as a complimentary for those who would like to get deeper understanding of the business perspective.      Later in October I'll start teaching again a course in the Technion based on the EPIA book (first one since the book is out, though I have used the book's draft for previous course).

On the duration of an event

I have neglected the blogging for a while, returned from my trip in Asia, planning for my next business trip to USA (I am travelling too much, I hope for a non-travelling period after that, but one can never know), and also took some days off for the Succot holiday.    Yesterday I traveled with most of my family to Tel-Aviv, to "Beit Hatfutsot", which stands for "Diaspora house", and documents the life of Jewish community over the history in many countries.    Here is an artifact from the exhibition:

There was also an exhibition of Andy Worhal painting notable Jewish persons,  one of the pictures is of Golda Meir, the only Israeli Woman prime-minister (time for the second one?) 

The VLDB conference also uploaded pictures from the conference, so here are two pictures - one from my tutorial, and the second showing me in the first raw (it was not really the first raw, but it was the first captured by the camera) listening to the keynote talk:

While I have been away there were some Blog posts by Paul Vincent that worth focusing upon, I have already commented briefly to this one, but want to have longer reaction about the issue of event duration that was raised by Paul.   


In most of the models events are considered as instantaneous, occurring within a single time point,  the temporal database glossary from 1998 puts "instantaneous" as part of the definition of event, the rationale is of looking on event as transition between two states, and transition in most models takes zero time,    A few years ago when we started the discussions about terms, I've pointed out the temporal glossary as a source for event definition, and David Luckham issued a strong objection to that definition, claiming that no event is really instantaneous,  even simple events like the "aircraft is landing" takes more than zero time, while events that are composed of other events - "complex events" - like the 1929 crisis (now we can talk about the 2008 crisis) is compose of many events and occurred over an interval.   


This is, of course, true, yet it is more convenient from computational point of view to deal with discrete time points than in intervals, furthermore, some systems have detection time semantics, looking at the time-stamp in which the event entered the system, rather than the time it occurred, this is the reason that we find time point semantics in most systems.


We can look at the following cases:

1.  The event really occurs within a time point,  e.g. time series of sensor measurements, or stock quotes.  There is indeed an interval among two successive events, but this relates to the state between the intervals and not the events themselves.
2. The event occurs within an interval, but the granularity of our time computation is bigger than the interval, thus we can approximate the interval to a time point.   Example:  the granularity we are interested is an hour, thus even if an event occurs within several minutes, we can still approximate it to the closest hour. 
3. The event occurs within an interval, and it is important to process it with an interval semantics, since we would like to see it relationship to another time interval (e.g. temporal context).
4. The event occurs in an unknown time-point that is bounded by an interval, there is some probability (e.g. uniform distribution) that it happened in any point of time within the interval.  In VLDB there has been a paper by Yaneli Diao and her students entitled: Recognizing Patterns in Streams with Imprecise Timestamps    Note that in this paper there are also some references to interval based semantics (of type 3).
5. Derived events are another type of events whose temporal semantics may be tuned.   For example:  the derived event "frustrated customer"  is being derived when a customer approaches a call center the third time about the same topic,  the question is whether the customer is frustrated only when approaches the third time, or the customer is frustrated over all the time since the frustrating event occurred until it is fixed. Furthermore,  derived event may also indicate an event that will happen in a future interval.   I'll write more about this issue in the future.

Bottom line:  the event processing systems of the next generation should support both time point and interval semantics along with uncertainties (Paul also had posting about "fuzzy patterns" on which I'll write in the future).