Fallacies in implementation of event processing solutions

Back in Israel... the flight that has taken me back home suddenly landed in Athens, instead of continuing to its destination, it turns out that one of the passengers left a laptop on a vacant seat and went to sleep elsewhere in the aircraft, some passengers attracted the attention of the crew, and after some consultations, they decided to land in the nearest airport to have security people check the laptop, after landing the laptop owner woke up and retrieved his laptop, and the aircraft could take off again (well - not so fast, took almost an hour to take off again) - strange story (has been in several variations in all daily newspapers in Israel)... a fallacy in (human) event processing system. Talking about fallacies, recently Hans Glide in his blog talks about "failure scenarios" where EP solutions may fail to produce correct results. This reminds me of previous work I have done in the area of derived data using active databases, where in a relatively old paper I have looked at several possible fallacies in active databases - the analysis is also valid for event processing, here are some examples:

1. non deterministic execution due to conflicting rules.

2. Inability to reconstruct the derived data from the raw data since multiple rules derive the same data-element

3. Redundant updates in case of inter-connected derived updates, e.g. the same data-element is updated several times within one path

4. Derivations may result in infinite loops

The paper also shows that it is possible to eliminate these problems using a specific execution model.

Taking it back from active databases to event processing - in active databases, this paper has been in the framework of deriving data, while in the event processing cases we derive events - however, some variations of the same phenomena can occur in event processing - certainly non deterministic behavior can happen, and some other fallacies as well.

Hand Glide mentions that each event processing implementation project should pay attention to these - and indeed event processing application debug is more complex than the debug of regular applications due to inter-relations between rules/queries/derivations.

IMHO - it is a responsibility of the EP vendors to provide semantic debuggers that will enable doing it. Some are already around, but more work is done in this area, the users should not be on their own here... more about validation of EP applications - in one of the next posts (I am promising many follow-ups, but new topics keep coming)...