In the last few days I spent part of my time in the NGITS 2009 conference, that was hosted by the IBM Haifa Research Lab, today the keynote speaker was Alon Halevy, whose picture you can see above. Alon gave an excellent talk about "searching the structured Web". As a database person that joined Google 4 years ago he discovered that many of the things he knew from databases were not valid in Google. and from data management point of view, he went several steps forward, and his challenge was how to process structured data that is hidden in the "deep Web" behind web forms and interfaces. This reminded me one of my colleagues in the Israel Air force, who spent a year of his life more or less (almost around the clock) to write a proprietary database system based on low level I/O operations, for some logistics process (I think it was automatic warehouse), he wrote his version for conncurecny control, query capability, recovery, and many other stuff. He thought at that time that the DBMS products available in the market (this was 1978 I think) are not good enough, and using them will be a step backwards relative to what he had in mind. I have written in the past about single application vs. more general one in the context of why network and system management guys have not developed more general event processing products. I was asked several times what is really the main issue behind all the work I am doing (along with many others) about "event processing" as a discipline, what is the new thing -- people have processed events forever. People also processed data long before DBMS has been introduced, but the way that my old colleague worked was not scalable. Not many people had the skills to do it, and it was not that cost-effective way.
Likewise, there were and still are many event processing applications of all types, colors and sizes, that are developed in an ad-hoc fashion, as there are still applications that process data that do not use DBMS products, because some aspect does not make it feasible or cost-effective.
However, it is clear that the DBMS area has made a tremendous contribution to the IT and business in the last few decades.
My own goal around event processing is to make event processing pervasive as part of enterprise computing, this will be achieved by generic software. Many of the database issues have been developed due the need for genericity -- take query optimization as an example, if one writes query by hand, it is not needed, since every developer can write optimized ad-hoc queries, the requirement to do query optimization came from the fact that a generic query language is being used for many purposes.
Personally, my interest is not in building "complex systems" (as the readers of my Blog know, I tend not to use the "CEP" acronym, due to the ambiguity of the term "complex") or one of kind systems. I think that the generic event processing systems will enrich their functionality over time, my interest is to make it pervasive. The first generation of products went some steps in this direction, and the next generations will do more. I have presented several times in various places my view about the next generation of event processing which refer to the challenges in the way to do it properly.
This topic will be discussed later this year in the 5th EPTS event processing symposium and the event processing Dagstuhl seminar planned for May 2010.
No comments:
Post a Comment