On messages and events

The smiling face belongs to Gregor Hohpe who is known as the co-author of the enterprise information patterns book. Gregor has conducted a BoF session in the EuroPLoP conference that I have written about in my previous Blog. The original book that Gregor has written has been about messaging oriented middleware, and now he is trying in his spare time to write the second volume - in the BoF he presented one chapter of the book talking about - identifying that something has gone wrong and handling it. This, by itself, is an interesting topic that I've been involved in the past on some related stuff, but as the title indicates the title of today's Blog is about messages and events. Hans Glide has recently dealt in his Blog in the issue of event processing vs. message processing, and part of the DEBS 2008 conference has been dedicated to messaging related research. Thus, this is a good timing to write a few lines about event and message processing --- being back home from all these travels.

Message is a transport mechanism, and as such message processing deals with the delivery of messages, relation to communication protocols, handling "not delivered messages etc..., one of the branches of messaging is also handling subscriptions when the protocol is pub/sub. The evolution of pub/sub has some similarity to that of event processing - started with subscriptions to every message published on a channel, moving to content-based subscription, which seem to be the focus of much work in the pub/sub research community. Content based subscription follows actually the ECA paradigm, where each message is treated as an event, thus, we see influence from the event world on the messaging world.

So - is event processing just fancy name to message processing ? -- not really, message processing is a lower level implementation that can serve also as infrastructure to event processing. An event can be represented by message, but not every message represents event. An event denotes something that happens, while I can send an old picture as a message and it is not an event at all. Thus the difference is in the level of architecture. Having said that, there are some interesting intersections: much of the simple and mediated event processing are shared with message processing (and indeed some of the patterns I am using to describe it are taken from Gregor's book).

The simple and mediated event processing are not new -- they indeed exist 15-20 years, and issue part of the functionality of event processing. The "detected patterns on history of multiple events" is the newer part in event processing, and it seems that it also a step in the evolution of pub/sub thinking.

Note that the pub/sub community calls itself "distributed event-based systems", and I indeed believe that they are part of the event processing community...

On EuroPLoP and Event Processing Patterns

This is Kloster Irsee in Germany, where the EuroPLoP conference is held. The conference itself is held in kind of an opposite way to other conferences, in a regular conference, the author of a paper presents, in this conference, the author of a paper hears other criticizing his paper, and does not talk. It also have some other properties -- some of the sessions are held in room which the chairs have been taken out, so people are sitting on the floor, or standing, there are also games embedded in the program --- my daughter would have said if she was here that there is a bunch of nerd children trying to behave in a cool fashion - well... anyway, we came here to combine work on creating a consortium for EU project and having a three hours "focused group" inside the conference, organized and moderated by Adrian Paschke. The audience was mixed - some were the EP gang, those who came to work on the EU project, and some where regular participants in the conference. The various interpretations of patterns related to EP have been discussed, and the need to advance in all of them. This is directly related to my tutorial in DEBS 2008, there is now work in progress on a pattern meta-language that we are doing in IBM, in a few weeks I'll tell you more about it - stay tuned.
More about my tutorial in DEBS 2008 - Paul Vincent from TIBCO has taken a picture of a slide that I've presented with different computer (due to some unclear technical problem) with probably different character set (Paul also blogged about this slide) - in footnote 1. Paul has been kind enough to send me the picture - so here it is, you may see some unknown characters...

More about patterns - soon.

On the multiple types of patterns in event processing

The red arrow in the map shows where I am now - in Kaufbeuren, a town in the south-west corner on Bavaria. I have arrived last night by train, and it has been a challenge to find a place to eat in 9PM, but with a few minutes of search I have found it. I am on the way to the patterns conference, in which we have a working group to establish a partnership towards EU project proposal. Thus, I would like to write a few lines about patterns. In my DEBS 2008 tutorial I have used the following slide:

In this slide there are four possible meanings for patterns in event processing:

(1). Patterns in the sense of functions that event processing application may perform (e.g. enrich, route, transform, filter, detect pattern).

(2). Patterns that are detected on the history of events (note that "detect pattern" is just one of the patterns of the first type, but the support of this pattern is what makes it CEP).

(3). Patterns in the user's domain - the way things are presented to the user (which may not be translated 1-1 to an implementation pattern)

(4). Patterns in the software engineering sense - best practices of how to use some language / product / technology.

All of the four are interesting and also have some dependencies, my tutorial has concentrated mainly on patterns of type (2), I have written about type (1) a lot in the past -- types (3) and (4) are frontiers that need to be conquered. More - Later.

On formal models and agents

This is an aerial picture of Karlsruhe, in Germany. It is morning, I have the habbit to get up early, so have some time for Email and Blogging (this hotel does have internet connection in the room!). Here I am a guest of the FZI team that is trying to apply semantic models to event processing, which is one of my recent occupations. One of the property of researchers everywhere (and IBM is not different) that they are constantly looking for research funding - since one of the available sources in this region is EU projects, researchers from the relevant countries (and Israel is part of the EU, for the sake of the R&D programs), so there is a need to join forces. The FZI group is looking at applying transaction logic as a formal model for both validation and run-time control of event processing applications. I am in favor of having formal model behind event processing to define precisely the semantics. A model that will succeed should also be relatively simple. I was asked whether I see a possibility to have a single formal model that inclues all variations of event processsing - and my answer is that my assumption is that this is possible.

Dealing with formal models - I have been asked by several people about the choice I've made in my DEBS 20008 tutorial to define "event processing agent" as an atomic unit of processing.

First - I agree that the term agent is overloaded, like many other terms -- event processing agent is not (necessarily) an intellignet agent, it is more a software agents. The glossary defines agent as a software module that processes events. I have making a distinction between conceptual agent and run-tine agent - the run-time agent may consist of multiple conceptual agents, and the decision is up to self-optimization process (will write about optimization issues in one of the next blogs). It is easier to reason (and optimize) when the abstraction deals with atomic elements. Whether should be called - "atomic agent" - may be. I prefer shorter names, and since this does not contradict the original definition. more - later.