Back to temporal databases

In 1998 I have edited a book of articles about temporal databases (together with Sushil Jajodia, and Sury Sripada), this followed a Dagstuhl seminar we held in 1997 about temporal databases, an area that was hot at that time in the research community, and somewhat cooled off.   Today a Master student I supervised took her final exam on "final work" (which is less than a thesis, a track that require to take more credit work), and did an implementation of a temporal database model from a paper in this book that was co-authored by Arie Segev, Avi Gal and myself.   This is somewhat more expressive model than the TSQL based models, and had its own interesting featured like: ability to freeze and unfreeze data, ability to distinguish between modification and revision, ability to deal with simultaneous value.  In fact some of these ideas found themselves into our work on event processing (e.g. policies when there are repeating events that may match the same pattern).  


Temporal databases as an area started in the Israeli army. Kobi Ben-Zvi who went from the Israeli army to do PhD in UCLA has invented the area, by formalizing the terms, and there has been a lot of work later in the research communities in the 1990-ies.    There was even big fight about how to extend the SQL standard to support temporal databases between two parties,  I don't really remember the details, in the book you can find the position of the two sides of this battle, as the Dagstuhl seminar was one of the battle fields.  The end result is that it never became part of the SQL standard, partly because of the fights, and more importantly since at that time the DBMS vendors have higher priorities on their mind -- e.g. Web related stuff, XML data etc..   There are some features, but it did not get fully into the mainstream of databases, although there are quite a few of specialized implementations.     One of the future directions of event processing will involve getting back to temporal databases as an infrastructure,   which is the area of retrospective event processing, I'll write more about it in the future.   

On Event processing as part of DBMS

Paris. I have arrived a few hours ago to Paris, and I went for a walk in the streets to stretch my legs after the flight, my hotel is not far from the Bastille, so I went there and watched the monument that you can see in the picture and the people who watch it.. Now, returned to my hotel to check Email and rest, before my hosts are coming to take me to dinner.

Today's topic is a short reply to some discussion that actually event processing should be done as part of DBMS, this is not a new claim, it is repeating from time to time by one database person or the other; in my past I have dealt with active databases that has attempted to put some form of event processing functionality as part of a DBMS engine, overall this approach has not lead to a big traction on the DBMS products. The main idea has been to add some language constructs in form of ECA rules (that also support composite events) to DBMS engines. The only traction on products from this works is the notion of "trigger" that does not really to justice to what the active database community has tried to do...

Anyway, twenty years have passed and the event processing thinking has been evolved from the early thinking on active databases. As said the main issue here is not performance, as some of the vendors claims, but TCO. Many of what is called "complex event processing applications" deal with the detection of patterns over multiple event instances and types, SQL may not be a natural language to express such patterns, in some cases due to its set-oriented thinking and some other limitations. In fact, in some cases customer reported that they could save 75% of the cost of development time by using language that can express patterns more naturally. This difference may not be materialized in languages that are by themselves variations or extensions of SQL, but this is only part of the EP universe.

Of course, the DBMS community can return to the idea of active databases and add language constructs to express patterns in the DBMS engine, and I guess that this may be a valid variation of event processing, but it will not naturally blend into SQL, it will have to be an hybrid language. More about this - later.

On the Gartner EPS 2008

Early morning in Stamford. The "hype cycle" has become Gartner's most known artifact, as well as their constant flow of TLAs - SOA, BAM, EDA, RTE, XTP - are all Gartners' words. The Gartner EPS has ended last night, and in less than two hours we'll start the EPTS 4th event processing symposium (so Roy Schulte will be able to relax and I'll start sweating).

Some short impressions from the conference (besides the networking and meeting again old friends).

• The Gartner analysts came with new slides, but very little new insights relative to their past messages.
• Mani Chandy had interesting talk, saying that EDA is a natural continuation of SOA and not a paradigm shift (good topic to discuss). Also said that one of the big benefits of EP is - saving time for people by filtering and aggregation of flowing information.
• David Luckham has talked about "holistic event processing" as the future -- which from technology point of view means - dynamic big event processing networks. Also talked about the need to have formally defined clear semantics of event processing language (well - that is what we are trying to do in EPDL).
• Marc Adler has a great talk (in my opinion, the best one) - about his experience in developing CEP application, which seems to be a success story. He talked about criteria to select a vednor, difficulties in the development itself, and shortages in the state-of-the-art. One of his insights is that unlike what the "stream SQL" fans are saying that since people know SQL anyway, it is a good basis, he claims that while the syntax looks like SQL, it is a totally different type of thinking, and the knowledge of SQL does not help on getting it.
• Richard Brown had a talk about collecting events from text - news, blogs etc... I think that getting events from unstructured data has a lot of potential, I also talked with somebody who told me about start-up that extracts events from video cameras - the scenario of tracing behavior of people that fit the shop-lifting pattern (was discussed recently in Blogs) - may become reality soon, it seems that the technology is getting there.

That is all my time permits me to write today -- more later.