On events in flight management

I have arrived home today from my Europe business trip - 11 hours later than planned. The crash in Heathrow airport had created a mess in the airport, only one runway operated, and my flight was delayed -
and I missed the connection in Zurich, and had to take the next flight to Israel - 11 hours later.
In the last year it seems to me that more events related to flights happened relative to previous years. One of the issues in event processing is - how should we react to occurance of event. In some cases the detection of the business situation is quite complicated, and we have been discussion "complex event processing" and such, however, in other cases, the detection is very easy, the complexity is in the response. Earlier this week, when I have arrived to Heathrow airport the following event occured:

The flight arrived 20 minutess eariler;

• The slot near the gate was still occupied.

The reaction has been - send the airplane to park outside the terminal and transfer the passengers to the terminal with buse. Soundes reasonable --- yes.

However -- it seems that nobody verified that there are buses available. The end result - we sat in the airplane 45 minutes, until the buses arrived. The captain told us several times that he is pushing them to send buses, but they are not responsive....

Back to last night --- I must say that I have been in this situation before, and that Swiss airline has handled it well - took the responsibility, gave us vouchers for hotels, breaksfat, and transportation from and to the airport. I have been in a somewhat similar situation last summer, when I had to fly Delta, and missed the connection in Atlanta, Delta's agents were very unpleasent, notified us that the vouchers for hotels we got at the source of the flights are all void, and that we are on our own. After 15 attempts I found a place in Motel Six, and got there by cab. Since I already had complains about Delta before - I have notified my travel agent to scratch Delta from the lit

The Swiss attitude in much better in ny eyes - but it seems that now I'll add another rule -- try to avoid connections at the last segment of the trip (going home !). I could re-arrnage my trip to do so. This is an intelligent event processing - mitigating predicted event.

Returning to normal writing of the Blog on more techncial issues - soon.

More thoughts on Rules in the context of Event Processing

I spent today (and tomorrow) in the IBM Hursley Lab - my second home in the last couple of years, this is a picture of the "Hursley House" (well - from the back side) - a countryside English manor that serves today as place for meetings and conferences, and the office of the Lab Director - besides this building there is a complex of connected building with multiple systems for room numbering that can provide in-door navigation challenges. I'll write today about some ideas that came out from discussions in the CITT meeting earlier this week about the role of rule technology. I still hold my opinion that although it is possible to take a technology that has been developed for one purpose and "hack" it to use it to other purpose, it may not be the most natural/effective/efficient way to do. This is true for SQL as well as rules when we are talking about pattern detection in complex event processing (I am working now on tutorial on the issue of patterns). However, this does not say that rule technology does not have a place in event processing in general. Here are some places it can be used:

(1). Decision-based routing in event processing networks.

(2). Transformation of events.

(3). Validation of events.

(4). Orchestration rules.

(5). Intelligent Event Processing.

Note that different type of rules are being used for the different cases -

for routing and transformation - it is typically - if-then rules/decision trees/decision tables.

for validation - constraint oriented rules.

for orchestration - ECA rules (note that orchestration rules are in the domain of the consumer that receives an event from the event processing network and has to decide what to do with it).

for intelligent event processing - all type of rules - deductive, inductive, abductive, rules with uncertainty - can play in different cases.

More about ECA rules and event processing - soon.

On situations

Hello from Zurich, got here in a train from Munich - and this is the famous train station of Zurich (in light), I have arrived in the dark, but has been in Zurich quite a lot of times before, tomorrow I am visiting the IBM Zurich Lab (which is actually located in Rüschlikon (a suburb of Zurich) home to four Noble prizes in physics. The hotel is 5 minutes walk from the train, but I forgot to print a map, and had to search a little. In the first half of the day I have participated in the continuation of the CITT conference, and earned my bread by giving a talk. After the talk I had some discussion with people about the term situation that I mentioned in the talk. I have borrowed this word from "situation calculus" in AI, and it fits well with terms in related domain such as: situation awareness. The discussion has been has been around one of the common misconceptions that situation is an alias to derived event. The answer is - no, these terms are not aliases ! Situation is some phenomenon that requires reaction (or notification). It is in the conceptual ontology of the consumer. Now - let's check the relationships with event:

(1). A raw event that is reported by a producer is a situation in the consumer's terms - in this case, only routing is required ("simple event processing").

(2). A raw event is reported by a producer has 1-1 correspodence to a situation in the consumer's terms, but more information or translation is required - in this case, the situation is created by mediated event processing.

(3). The situation occurs in a deterministic way once an event pattern is identified, and it is a functions of the events participating in the pattern - in this case the situation is created by complex event processing.

Note that the situation is not the derived event - but the derived event in this case is an indication that the situation is detected, note also that not all derived events are situations, derived events that are consumed by other agents, are not situations, only those that are consumed by event consumers.

However, situation detection may not be purely deterministic - and here we are getting to "intelligent event processing". Example: the pattern is only an approximation, example: the pattern is - "an event E1 happens at least 4 times within an hour". Now, if the event E1 happens 3 times and the pattern is an approximation, there is some probability that the situation did occur. This is even before looking at probabilistic events.

More about uncertainty in event processing - later.

CITT BAM-BPM-CEP-EDA conference Day 1

This is another picture from Regensburg - a picturesque medieval town on the Donau. Today I have attended the first day of the CITT conference organized and navigated by my good friend Rainer von Ammon .

The conference is held in an old historical building, and like the VLDB that was held in a historical building in Vienna, the architect of that building also is in the opinion that people love going up and down the stairs multiple times... Anyway -- I have learned several interesting things - one that is not really related to the conference-- one of the speakers revealed that there is a formula to calculate the value of any digit in pi without calculating the other digits - and indeed after some search in the internet (wikipedia does not seem to know about it) I have found it .

Other than that - some discussion about creating a consortium for an EP European project, some presentations about "fraud detection", a keynote address was given by Stefan Reid from Forrester Research - the topic has been - " ESB Open Source Challenge", and he discussed what is ESB and is it mature enough to be an "open source" topic. His ESB definition has been too broad to my taste (i.e. he sees business process modeling and monitoring as part of ESB, I think that they are not really ESB functions, and architecturally they are in another category). He also said that "in the end of 2008 any ESB that will not include CEP will be obsolete" - and I obviously like this statement. I think that ESB (in the SOA case) is the major (but not the only) carrier of CEP (but I'll discuss ESB in a seperate posting). Later in the day Rainer brought his entire class of students and they presented BPM oriented projects. The last presentation was done via Skype from Redwood City - two of Rainer's students that are doing a project in Oracle has presented their project, while the two CEP Oracle champions - Dieter Gawlick and Shailendra Mishra (my EPTS partners), have presented the upcoming Oracle CEP. Well - there is some room to improve the skype technology.... There was also in the middle a panel about BPM patterns - and somebody made the observation that people talk about technical patterns, while there is no enough work on business patterns - this is true in a way for EP as well, but I'll discuss patterns in another posting. Tomorrow - second day, where I'll have to earn my trip by giving my keynote talk entitled: "event processing as part of enterprise architecture" - well, since I am working in a "just in time" mode - I'd better finish this presentation.

PS.

Tim Bass complained in his popular Blog about the lack of possibility to read blogs and from China -- I have checked in the log of my blog (thanks to google analytics) -- it seems that there are (not many but some) people form China that are reading my blog - I even have one reader from Xiamen (most are from Beijing), so probably there is a way to overcome it. I have done a business trip to China (Beijing only) in 2004, and it has been very interesting experience, indeed when I watched CNN in my hotel, and they started to talk about demonstrations for democracy in Hong-Kong, there has been a sudden communication obstacle that has been resolved three minutes later, but this has probably been a conicidence.

I am also glad for Tim that democracy has been restored in Thailand after some period of military control.... things are fragile in our universe... Back to my presentation now - more later.

How events are getting into processing ?

Hello - again in a business trip after some break in trips - now in Regensburg, Germany (see in the picture) for the CITT conference that starts tomorrow, so I'll have time to write about it later this week. The topic of this posting relates to the event processing conceptual model and deals with the producer side.

Getting events from a producer may be the most time-consuming work in the event processing application, depending on number of producers, existance of adapters, and complexity of instrumentation (if needed)

A producer can be either a sensor that reports on an event it senses, or a creator of an event, in which case the event is typically an instrumentation of something that happens within the producer.

While "event-driven" is closely associated with the push mode - meaning the consumer decided when and how it sends events, this is only one of the possible modes. In some cases "pull" mode is used - meaning the event processing networks pulls events from the producer.

Why is pull needed? several reasons:

(1). Not all the sources are really producers -- the source may be a piece of legacy software that has not been designed to produce events.

(2). Push may not be cost-effective - we have to weigh the benefit of processing events fast, and the overhead occurs in transaction systems to obtain the events. This depends on the application, on type of processing for events and the tolernace of the operational system.

Pull can be done as: periodic pull or on-demand pull. More about the conceptual model parts - later.