My last trip abroad was a mix of vacation and business trip (to attend the Tretno event processing symposium, and the meeting of the EASSy consortium that is writing a proposal for EU project), as such I took a piece of luggage with me, and deposit it (in Hebrew the words "to deposit" and "to abandon" are very close --: "lehafkid" vs. "lehafkir", thus I am typically saying that I have abandoned the luggage in the airline...), anyway, standing near the carousel (something I really hate to do !), I realized that I am actually doing a pattern matching. I have a mental model of my luggage in my mind (shape, color, other characteristics), and I am trying to match this mental model with the pieces of luggage flowing on the carousel. There may be more than one that fits my mental model, and sometimes I grab one, look closely, and decides that this has been a "false positive". Likewise, we are matching patterns many times in our daily life.
So - what exactly is an event processing pattern? In the EPIA book, we have dedicated a (long!) chapter to pattern matching in event processing, and started it with the following definition:A Pattern is a function that takes a collection of input event instances and produces a matching set that consists of zero or more of those input events This is, in essence, what we do in the luggage case, selecting one out of a set. Interesting cases are cases in which we should find events that the pattern relate to their combination (this, according to some definitions, what makes the "Complex" in the "Complex Event processing" pharas), so they need to satisfy some pattern as a collection of events, and not as individual events. An example is we wish to determine if a moving object (say: motorcycle) is moving north, by looking at the location of the motorcycle over some period of time, we can observe whether the progression of location satisfies the pattern of "moving north". I'll write more about pattern types soon.
Today is a happy day to the Israeli scientific community it was announced that Professor Ada Yonath from the Weizmann Institute won a Nobel prize in chemistry, this adds to two Israeli scientists that won a Nobel prize in chemistry a few years ago, so Israel is a chemistry super-power. Computer science was not exist when Nobel decided on prizes, the equivalent is Turing Awards, that, if I am not mistaken, have been already awarded to three Israeli scientists, so we are not bad in this area either..
Back to event processing thinking --- getting progress on the EPIA book, yesterday we had two hours review with the editor on editorial stuff around the last three chapters, and upon revision, it will get to the 2/3 review. Our target date for publication is now April 2010, and this will probably be the final target.
Anyway, to continue my previous posting on contexts, I would like to discuss the notion of context composition. Recall that context is grouping of events based on one of the following: time, space, state and segment, for either grouping in order to apply operations on this group, or make distinct behavior for distinct groups. Composite context -- as its name suggests is just a multi-dimensional context, i.e. it contains cross-section of several contexts.
In the picture below this is a composition of segmentation ("per customer") and time (in this case fixed sliding temporal interval - per hour, each square is a context partition.
This is just example of combination, there are other useful compositions such as: spatio-temporal context: partition with one dimension is time and one is location oriented, or space and state contexts combination, example: spatial context == within the city of Trento, state of weather = {sunny, cloudy, rainy, snowing}, in each of these partitions other agents are applicable.
Somebody asked me -- what is the benefit of using the context abstraction anyway --- the answer is, like any other abstraction -- it saves work. The same application can be written with much less code and is much simpler to develop, maintain and understand --- the use of context is quite useful in this sense, see also some discussion by Marco. Next -- I'll write more about the next chapters on event processing patterns, stay tuned.
In the last few days, dishes in my house are washed by hand. Our dish washer have seen signs of old age, thus I acquired a new one. First step was to order delivery from the store. There is the software engineering principle of "separation of concerns", thus those who do the delivery don't install it. I am not allowed to install it myself, since if I open the package I am losing the warranty, only a technician of the service company related to the importer is authorized to open the package. After some coordination this technician arrived today, looked at our kitchen and said -- "this is very common, they have changed the standard, now there is a thermostat on the pipe, so the size of the hole that your need in the kitchen closet is larger". Since there is a separation of concerns, he does not do holes, and I had to call the Carpenter who came later today, so I'll be able to call the technician again tomorrow... This is the result of the fact that the event of "standard of pipe changed, thus it effects the size of the pipe" was not reported by the store, since they don't install dish washers so they probably don't know --- this separation of concerns, and not seeing event over a larger story, wasted a lot of time and money to the society...
Anyway -- enough complaining.
I got a question from Hans, related to my posting on context, I am copying the question here:
" I would be interested to hear about how the context concept works in a particular use case: Let's say I have events that contain, among other things, a number. I would like to capture the third quartile (a data point that sits immediately above 75% of the other points) of the number, every hour. I would then like to perform some operation over the previous 8 of these hourly numbers. How would this be expressed as contexts and agents?"
OK -- here it is:
We start by event of type E1 that flows into the system from some consumer and has some numeric attribute A, for which we want to capture the third quartile.
- First we define a context C1 of type of sliding fixed temporal interval with both duration and period of 1 hour.
- Then we define an agent A1, which is valid within C1, and calculates the third quartile and produces event of type E2, with an attribute B which that captures the derived value.
- Further we define a context C2 that is a sliding event temporal interval on event E2, with event count period and event count duration both of 8.
- And finally we define an agent A2 that subscribe to events of type E2, and is valid with context C2, that does some operation on E2, and may in turn produce (say) event of type E3, which is flowing to some consumer.
This is the basic model. Of course, it is a bit more complicated, if our agent library does not contain an agent that calculate quartiles may not be a basic, this can be somehow combined from some combination of agents, since an agent can be recursive and include mini-EPN, but as such we can still model it as a single agent in the high level view.
The nice thing about this abstraction that it is quite simple to model such problems... More - Later.
