Towards an event processing manifesto - take one

The word manifesto brings to many people an association with the Communist Manifesto by Marx and Engels, which is an example how manifesto can be used, reused and abused. The term manifesto has been adopted by the computing community, and we have seen variety of works entitled: manifesto, some of them are: object oriented database manifesto, the manifesto for agile software development, and the business rules manifesto - just to name a few examples.

In May 2010 I am co-chairing (together with Mani Chandy and Rainer von Ammon) a Dagstuhl seminar, whose task is to compose "event processing manifesto", which we strive to make it as influential effort. A Dagstuhl seminar is a five days retreat in an isolated place in which a selected group of people are convened to make deep discussions about a certain area in computer science, many of these seminars are geared towards the academic community, but it this particular one we strive to keep the balance and have half of the participants from industry (vendors, independent consultants, analysts, and some innovative users), the number of attendees in each seminar in limited, and there is a big competition on this resource (and long waiting list after approved, we have put the application to do it in 2008). Here is a picture of Schloss Dagstuhl (located in Saarland, Germany):

I guess that I'll write a lot about the Dagstuhl seminar, the event processing manifesto, and related stuff in the next few months, so this is the modest starting point.

The intention of the manifesto is simple (but not easy) -- define what event processing is, what is the scope of event processing, are there mandatory features in event processing, what are the various options and variations, what are the synergies with other disciplines, and have as an appendix a call for action for the community: what are the research challenges and what are the pragmatic challenges.

I'll start today with some of my own thoughts on the scope issue.

I view "event processing" as a name of a computing discipline like "data management", "image processing" and "information retrieval" - relatively wide disciplines that cover wide collection of applications. This is also true for event processing. Alas, we still have some misconceptions even on the scope, I have used before the metaphor of blind people touching various parts of an elephant and concluding that each of them has identified what an elephant is, so this time I'll use another animal - ant whose sight is limited to its immediate environment.

I am using ant, since one of the great Hebrew poets has written a poem saying (in free translation): "my world is narrow as the world of an ant". Some examples of it:

1. people who view event processing as aggregation of time-series events, this is definitely an event processing application, but not the only one;
2. other people are looking at detection of anomalies within event collections, where the anomalies types are not predefined as what event processing is all about, this is indeed another event processing application, but not the only one;
3. yet other people are looking at real-time matching of predefined patterns with streaming events, which is somewhat different from the two previous examples.

One might claim that there is no event processing discipline at all, but there are very distinct areas that process events in different ways for different purposes, probably using different technique; while this may be a possible conclusion from the Dagstuhl meeting, my experience shows that there are many applications that require blend of event processing functionalities, and that the space is not discrete, but has some continuity, thus event processing in the wide sense should deal with all of them (and some more) and the synergies between them. So this is a point for discussion.

While the amount of people that will be involved in the actual composition of the manifesto is restricted, we'll strive to disseminate the draft to a wide discussion, both within the EPTS community and other avenues. As a comment, the Dagstuhl seminar is not directly related to EPTS, many of the attendees are not EPTS members, and some are members of adjacent communities and not the event processing community, however I think that EPTS should have a role in follow-ups.