On Event processing as part of DBMS

Paris. I have arrived a few hours ago to Paris, and I went for a walk in the streets to stretch my legs after the flight, my hotel is not far from the Bastille, so I went there and watched the monument that you can see in the picture and the people who watch it.. Now, returned to my hotel to check Email and rest, before my hosts are coming to take me to dinner.

Today's topic is a short reply to some discussion that actually event processing should be done as part of DBMS, this is not a new claim, it is repeating from time to time by one database person or the other; in my past I have dealt with active databases that has attempted to put some form of event processing functionality as part of a DBMS engine, overall this approach has not lead to a big traction on the DBMS products. The main idea has been to add some language constructs in form of ECA rules (that also support composite events) to DBMS engines. The only traction on products from this works is the notion of "trigger" that does not really to justice to what the active database community has tried to do...

Anyway, twenty years have passed and the event processing thinking has been evolved from the early thinking on active databases. As said the main issue here is not performance, as some of the vendors claims, but TCO. Many of what is called "complex event processing applications" deal with the detection of patterns over multiple event instances and types, SQL may not be a natural language to express such patterns, in some cases due to its set-oriented thinking and some other limitations. In fact, in some cases customer reported that they could save 75% of the cost of development time by using language that can express patterns more naturally. This difference may not be materialized in languages that are by themselves variations or extensions of SQL, but this is only part of the EP universe.

Of course, the DBMS community can return to the idea of active databases and add language constructs to express patterns in the DBMS engine, and I guess that this may be a valid variation of event processing, but it will not naturally blend into SQL, it will have to be an hybrid language. More about this - later.

Some footnotes to Luckham's "short history of CEP- part 3"

David Luckham, who took upon himself to survey the history of "complex event processing" has written a series of articles about the roots and developments the CEP area, while this is relatively young as discipline, it has roots in other disciplines, and people who worked separately from different perspectives, origin disciplines, and types of applications.

I recommend reading it. I'll make just some small comments for the recent third article :

(1). One of assertions in the article states:There were research groups within these companies engaged on similar projects to those in universities. Some of the groups ended up producing
prototype CEP products. In these cases, the decisions as to whether to let these
prototypes go forward to productization were haphazard, probably because they
were made by a management that didn’t understand the technology or its
potential.

Well - one must realize that corporate executives have infinite amount of wisdom, otherwise they have not been corporate executives, and if a decision looks haphazard, it is only due to the mental limitations of us, simple mortals.

(2). Another assertion is: But the largest number came out of an active database background. This is the reason why many of the commercial languages for event processing are
extensions of SQL
More accurately -- some of the current products we see are indeed descendants of "active databases" which was based on ECA (event condition action) rules. Among the products which apply this approach we can find RuleCore and Aptsoft (now Websphere Business Events); the products which extended SQL came from different paradigm in the database community of "data streams management" according to which, the queries are constant and data flows, instead of databases in which the data stands and queries flow. This has been later development which started with Jennifer Widom's Stream project; the two paradigms, though both came from the database community should not be mixed (although there are certain individuals who were involved in both).