Tuesday, September 11, 2007

Event Processing - A paradigm shift ?

In the Dagstuhl seminar we held earlier this year Roy Schulte has raised the question: "Will Event Processing (EDA) become a paradigm shift in the next few years or not?”. If my memory does not mislead me we have discussed it in one of the evenings in the wine cellar, that's the reason that I don't remember much from that discussion. Anyway, thinking back about this issue and looking at a recent article claiming that "complex event processing is still on the launch pad"
we can realize that the paradigm shift has not happened yet (maybe it has happened for us which live and breeze this topic, but not in a large scale anyway). Looking at one paradigm shift that has succeeded - relational databases - we can analyze some of the reasons for that success :
  • We need an underlying theory in two areas one - semantic (like relational algebra) and the second engineering-oriented theory (like query optimization) that is built on top of the first one;
  • We need vendors that understand the theory to generate products that implement it
  • We need to be able to explain the developers community (with the various types of developers - a topic for another article) how to use it --- a good textbook like Date's book about relational database is a good step, but development of use patterns, methodologies etc.. will be helpful too...
  • Standards (topic for another forthcoming article) are complementary --- but relational databases have been paradigm shift before the SQL standard was published.

The state of the practice in event processing is similar to the database area in the pre-relational time, there is several approaches, all of them grown up from implementations. This does not at all undermine the importance of the first generation of event processing products, without more experience in the field, we cannot get anywhere...

While the various vendors continue to incrementally advance the products, Some of the effort (perhaps a community effort) should go now towards bridging the gap between the first generation and the "paradigm shift"...

Bottom line: Event processing has a big potential to make the paradigm shift that Roy Shculte is talking about, making event processing a major paradigm in enterprise computing. It can happen, it should happen, and we should make it happen - but there are mountains to climb and oceans to cross.

More on the challenges and obstacles as well as the futuristic vision - later

No comments: