On DEBS 2010 tutorials day

My role in the organization of DEBS 2010 was to handle the tutorials day. Starting in DEBS 2008 in Rome, the tutorials day became (for me) the most interesting day of the conference. Today the tutorial webpage is up and running (this is a first version, some missing details will be added soon). Some of the tutorials have been submitted in push mode by their authors, and some have been solicited in pull mode by me.

Six tutorials are planned: three tutorials in parallel in the morning, and three tutorials in parallel in the afternoon.

There will be two tutorials provided by EPTS work-groups, this time:

• A tutorial about event processing architectures given by Adrian Paschke and Paul Vincent who lead this group
• A tutorial by the new work group that deals with the value of event processing to customers, this tutorial will be given by a collection of people. The workgroup leaders are Rainer von Ammon, Guy Sharon and Nenad Stojanovic.

The additional four tutorials - two of them can be thought as extensions to the event processing languages tutorial that Adrian Paschke, Jon Riecke and myself have presented in DEBS 2009 (and was viewed or downloaded by 3551 persons, last time I checked):

• A tutorial about SQL-based event processing, given by Bernhard Seeger.
• A tutorial about Logic based representation and reasoning of event recognition, given by Alexander Artikis et al.

The additional two planned tutorials are:

• A tutorial about event processing in wireless sensor networks that was delayed from last year since the presenter Antonio Lureiro had to cancel last minute
• A tutorial that was prepared with the help of our team here in IBM Haifa Research Lab that will deal with context aware computing and its utilization in event-based systems. The tutorial will be given by Ella Rabinovich (who will be very active in the conference as she will present two papers in additional to the tutorial) and myself. Additional of our team members that helped in preparing the material are: Yonit Magid, Inna Skarbovsky and Nir Zolotorevsky. I'll write more about this specific tutorial closer to the DEBS conference.

The DEBS research and industry tracks program chairs have also sent notifications about rejection and acceptance this week. I think that there are 21 papers accepted (16 in the research track and 5 in the industry track), the research track acceptance rate was 25%.

Our lab is very active in this conference (various people on various aspects), among the 21 papers there will be 5 papers that with all or some authors from IBM HRL, and it will also have co-authors on 2 of the 6 tutorials mentioned above. This is an indication to the focus that the event-based systems receive in the lab.

Preparing for the Dagstuhl seminar on event processing

The countdown for the Dagstuhl seminar on event processing is now 17 days, and we are preparing for that event. Schloss Dagstuhl, seen in the picture, is a castle in Germany, which hold a 5 days events that deal with notable topic in computer science, for each event there are 4-40-45 invitees, most of them are the leading figures in the area, and some are young promising researchers in that area. The Dagstuhl seminar that I'll be co-chairing with Mani Chandy and Rainer von Ammon, is planned for May 16-21. Hopefully there will not be ash clouds in Europe at that time. As I have written before, the idea is to focus on the event processing manifesto. We have now determined that the five chapters of this document (or five appendices, since the manifesto itself will probably be a one page document) will consist of five chapters that deal with the following five topics:

Topic 1: Event processing scope, classification and business value.

Topic 2: Event processing functions: present and future, common and additional functions.

Topic 3: Event processing and the rest of the IT world: relationships of event processing with other areas (databases, rules, BPM, analytics, cloud computing, social computing…).

Topic 4: Event processing standards: What standards should be done and when, what should be the starting point and roadmap in each standard.

Topic 5: Event processing grand challenge: What we would like to achieve if we can get a considerable worldwide research investment? What are the research goals? What are the means to achieve? What will be the end result of this research (e.g. what scenarios will be able to be dealt with?)

Each topic will be analyzed by a group of participants, and discussed with the entire set of participants. The end result will be a clearly articulated document that deals with the scope, analysis, and future of event processing, and seed a substantial federated research projects.

Besides the professional side, it will also be a good opportunity to meet old friends that I don't see on regular basis. Stay tuned to more on the Dagstuhl seminar (and I am sure that some of the event processing bloggers like Marco Sierio or Paul Vincent will provide you other perspectives about this seminar.

Revisiting Web 2.0 and event processing

To those who like interesting numbers -- I've noticed this week that the number of my contacts in LinkedIn reached 777, it will probably not stay long, as it keeps changing. I have written before about the use of event processing in social networks, an area that I think has big potential that has not been fully explored yet. I have challenged the students in my event processing class at the Technion to select a project that is somehow related to the use of event processing in Web 2.0, and social networks in particular.

I have written before that unlike the phrase, event processing is not always about threats and opportunities. In fact, typically in Web 2.0 the use of event processing is in information dissemination - getting the right information to the right person at the right granularity in the right time. This is a case of situation awareness while the situation here is the fact that some website content has been changed, or that some activity related to social network, Blog, Wiki, and other collaborative applications happened. This can have any of the event processing functions --- filtering seems to be quite dominant, but also transformation - aggregation, translations, splitting all have a role, and of course pattern matching may also have a significant role -- example: send me update after three changes, patterns related to number of readers in a Blog (Google Analytics provides some), patterns related to activities in social networks over time and space, patterns related to tags in twitter and more... I think we'll hear much more about this topic.

Letter to the editor in the Communications of ACM

Paul Vincent attracted our attention to an article in the Communication of the ACM written by Julian Hyde from SQLstream, and wondered about some of the assertions that were expressed in this article. Reading the original article, I indeed found some inaccurate assertions that somehow missed in the review process of the CACM magazine. After communicating with the editor-in-chief of CACM, he suggested that I'll write a short response (and the author will have a right to respond to the response). It takes some time, but today I got the electronic copy of CACM and found that my response, and the response of Mr. Hyde to my response have been published. You can read for yourself (under the title - "event processing anywhere"). In essence I have mentioned three inaccuracies in the original article:

1. On whether event processing is a blanket term for stream query systems: actually event processing is much broader term then stream query

2. There is a religious war between SQL and non-SQL vendor. I have not noticed the religious war, many vendors have both SQL and non-SQL interfaces.

3. Event processing is restricted to one application type in the financial services area, while other application areas are neglected; this is indeed a common misconception, and I have written about it in the past.

Mr. Hyde took his right of response by saying that the event processing vendors fail to satisfy the expectations of the BI community, this is a new claim, which does not seem to respond to any of the issues above.

Paul Vincent in his Blog's post raises another interesting assertion from the CACM article that streaming query engine is a new technology -- well, new is a relative term.

Bottom line: while people are used to view marketing material with a grain of salt, it is typically expected from a respectable magazine which employs review processes to be more accurate, since people tend to take it as a reliable source.

On the production phase of the EPIA book

I have co-edited some "collection of articles" books that are common in the research community, there is certainly some logistics associated with it, but relative to producing a "mass market" book it is a children's game comparing with the production of the EPIA book that I have been writing together with Peter Niblett. The publisher of our EPIA - Manning, has various ways to ensure quality. Manning is a publisher focusing on computing related book, thus it views book development as a software development project (and indeed many of its books are code-driven). While the book was initiated by Manning and not by us, they have sent the outline we sent to bunch of reviewers, all of them people with deep knowledge in event processing. Some of the advices we got were very useful. Since we wrote in the outline that we are going to base the book around a single example, the advice of one of the reviewers was that we'll use an example that everybody can understand, the reviewer added that if we would chose an example from the financial services domain (as many of the EP papers are doing), we might create a communication obstacle with some of the readers, who will not be familiar with the terms. We followed this advise and created the "Fast Flower Delivery" (FFD) that already received multiple implementations in many languages. For each 1/3 of the book, we had a milestone in which they sent the book to a bunch of reviewers (in one of them there was a response from 14 reviewers). The reviewers were mix - some people who have deep knowledge in event processing, and some who don't know what it is, both types of reviewers sent various types of comments, some of them resulted in restructuring of the book (originally we planned 15 chapters and 3 appendices, we ended up with 12 chapters and 2 appendices, but with around 100 pages more than we originally planned)- This was just the development process, and it last for 15 months. A few weeks ago we have started the production phase -- it has a workflow (supported by electronic content management system): Another technical review by a "technical proofreader" - here we looked at PhD student in this area, and thus addressed faculty members active in this area; the person who did this job is Samujjwal Bhandari, a PhD student in Texas tech university, he has read all chapters, made comments and caught various cases of inconsistencies among the different chapters, then it moves to the copy editor who makes editorial modifications, and returned to us with many comments and questions that we had to answer, after doing this round, each chapter is going to the proofreader, who is doing another editorial pass, and then comes back to us with questions and comments, so we are doing another pass on all chapters. After that it goes to the typesetting. In the background there is a graphical work to redo our amateurish figures in a professional way, and then hopefully the book will be ready. Currently we passed the phases of the copy-editing, and now working with the proofreader. There are a lot of editorial rules, and style issues that have to be dealt with (Peter is much better than me in noticing the small details), and in fact -- amount of work is much higher than I anticipated, it consumes much of my free time for over a year now, as we both are doing it in addition to our daily work -- but I hope the result will justify the investment. More about the book - later.

On computer science research publication

I have read one of the recent issues of the Communication of the ACM - as ACM member I am getting the hard copy of this magazine and from time to time I am browsing through it. One of the articles that caught my eye is an editorial article by the editor in chief, Moshe Vardi. It summarized some opinions about publications in computer science, it turns out that most disciplines make journals as the primary vehicle of publications, and view conferences more as a social gathering where most submitted papers are being accepted, while in the computer science area, conferences are a very dominant publication vehicle, and as such, the "good" conferences are very competitive, in fact, people are writing in their CV, what is the acceptance rate of each conference they published in. Vardi's article shows support in opinions that computer science has to mature, and behave like all other disciplines. This observation is true, in my previous incarnation, as faculty member in an inter-disciplinary school at the Technion, I quickly discovered that other disciplines don't really understand the term "refereed conferences", and that conference publication "does not count". One of the wrong (in my view) academic metrics is to count papers, but this is not the topic I am discussing now. From the industry research perspective, the current state of conference-centric, is very comfortable. Since (unlike our academic colleagues) writing papers is a secondary occupation, then it is comfortable to write relatively short papers based on the conferences's page restriction, it is comfortable to tell management that you have a deadline for publication then to dedicate time for undetermined deadline that always gets lower priority to other things that have deadline, and it is more comfortable for management to let the reviewers do the filtering of who should travel to conferences. Despite these facts, I think that getting computer science to act like a mature discipline is the right way to go forward. The journals and conferences will need to be adjusted. Academic journals will also are also going through a phase of coping with the Internet era, and various business models for journals are emerging. The event processing perspective of it is that we should consider to found an event processing journal, when we feel that we have enough quality content for it. The numbers of submissions to DEBS are indication that critical mass might exist, I guess that this will be discussed in some of the coming meetings (like the Dagstuhl seminar),

On virtual vs. real in event processing

As a follow on to my last posting on virtualization of event processing, one of the frequently asked questions, and maybe somewhat open question can we get really to platform independent models that can be directly compiled to implementation of our choice in fully automatic way. This is one of the major topics I am investigating recently. Two issues are involved: Is it possible to automatically compile to a specific platform that may be semantically incompatible with the platform independent model, the second issue is whether we can create an efficient implementation without taking more platform specific considerations. The answer lies within the compilation process from the platform independent model into the specific platform. This was not really done in event processing yet, but problems of the same magnitude have been handled in other areas, and we intend to leverage the experience of people who have done it in order to see how far this can go. As a related issue, Jim Odell continued his series on event processing and agent technology, hosted in the TIBCO CEP BLOG .

Since the platform independent model we are using has "event processing agents" as its major building block, agent oriented implementation is the most natural one, and the compilation to such a model will be straightforward, however, the platform independent model should also be compilable to centralized implementations, where the agents are all hidden inside a single run-time artifact. My guess is that the shift to agent oriented implementations will happen gradually, I'll talk about it when I'll present my view on "event processing - seven years from now" in the OMG event processing consortium meeting next month. I'll write more in the future on the compilation from model to implementation as we'll get progress in that project,