Wednesday, October 17, 2007

on event processing language and modeling



Hello From the IBM Hursley Lab in UK, in the picture you can see the historical building known as "Hursley House" where customers' meetings are being held, where I spent most day today...

In previous posts I have argued for the need for a specialized event processing language since the functions of event processing are specialized, and on the other hand in another posts I have argued for event processing being part of a bigger whole - which argues for more general programming language that can express the "bigger whole". This is clearly a trade-off here.

What can be done to "have one's cake and eat it too", have the specialized language but also remain within the more general framework -

How this can be achieved - some ideas:

  • Have a unified meta-model that can generate multiple lower language languages
  • Have an embedded EP language inside regular programming language the way that are embedded SQL work
  • More ideas ?

This is not just an event processing discussion but leads to larger discussion about specialized vs. generic tools, and the trade-off involved.

Monday, October 15, 2007

on the various temporal dimensions of an event



This is the Winchester cathedral in England, and I am finding myself in additional business trip, but this time to the country-side of England, one hour drive from London, and seems like several centuries distance in time. Talking about time - I'll dedicate this blog to temporal issues in event processing. Well, this is a big topic, so I'll start with the topic of the temporal dimensions of an event. The first issue is - does event happen over a time-point or over a time-interval. A couple of years ago I had a discussion about it with David Luckham
who claims that "nothing happens over a time point", which means - everything happens over an interval. This may be true, but for practical reasons, there are cases in which we want to look at an interval as a time-point, since we are not really interested to decompose the interval further, the abstraction of chronon as atom of time is very useful - saying that we can look at the "smallest significant unit of time" for a certain application, examples: minute for delivery of packages, 1/100 second for athletics, day for personnel applications, month for salaries etc...
Using this we can say that an event occurs over a time-interval, but for some (maybe most) applications it is sufficient to approximate the interval to a time point in the right granularity (the application's chronon). However, this is talking about the time that the event happened in reality - indeed there maybe several temporal dimensions associated with a single events:
  • The "event occurrence time" - the time that the event happened in the reality
  • The "event detection time" - the time that the event was emitted / signalled / detected by the event processing system (typically a single time-point).
  • The "event valid time" - the time-interval in which the event may effect the event processing system - typically starts at the time it starts to effecting the processing, and ends at the time it stops effecting the processing. This is less obvious then the other two, but important for some types of processing.

In some cases the order of events do not matter, however, when it matters - e.g. a pattern is depends on the fact that event e2 occurs after event e1 (such as: the order of the events "printer is offline" and "printer is online" of course matters, since it indicates whether the problem has been resolved or is still there. The order in this case is of course according to the "event occurrence time" (note that if the event occurrence time is an interval, there may not be a total order, but interval relations).

This is only a start of the temporal saga in event processing, there are many more questions that I'll try to include in future blogs:

  • What happens if events are getting out of order ? (short and long delays) ?
  • What is the "occurrence time" and "detection time" of derived events ?
  • What is retrospective event processing, and how it fits in ?
  • What is temporal context and how it relates to time window ?

More on all of these - later.