Tomorrow night I am planned to start the long way to Nashville, to attend the DEBS 2009 conference. DEBS is originally a conference founded by the distributed computing guys, what is known as the "pub/sub" community. Two years ago, Mani Chandy and myself came to the DEBS steering committee and challenged them to extend the conference such as it will be a general event processing conference, and work with them to make it the scientific "flagship" conference of event processing. Looking at the DEBS technical program, the first day (Monday) is the tutorials day. It will have six tutorials, five of them are about event processing, and two out of the five were created by EPTS work groups: the use case work and the language analysis work group. Giving part of this tutorial (the other parts will be given by Jon Riecke and Adrian Paschke) will be my first role in this conference. We shall make these tutorials available to the public after the presentation. The second day (Tuesday) will start with a keynote address by John Bates, Apama GM, and one of the pioneers of the event processing area. He will be the industrial keynote speakers, and as chair of the industry track, I will have the honor of introducing him, my second role in this conference. The first research session on distributed event processing will have a paper which I co-authored on stratified approach for supporting high throughput event processing applications, however, I will leave the presentation to the younger generation (Geetika Lakshmanan from IBM Watson Research Center will give this talk). The second research session will be about pub/sub, but one of the talks has the event processing phrase in the name. Then we'll have the industrial session, and my third role will be to chair it. This session will consist of four industry report, and industrial panel about academic-industry collaboration in event processing. We plan to have four panelists, two from academia, one customer, and one vendor, and discussion with participation of the audience. DEBS originally was almost purely academic conference, but we have injected some the industrial participation, and believe this partnership is important for both sides, and more important, for the future of the event processing area. In the DEBS business meeting, later that evening, I plan to propose hosting DEBS 2011 in Israel. The third day, Wednesday, will have a keynote talk of Alex Buchmann, an old colleague from the active database era, who is one of the DEBS founders. The research session of that day will deal with complex event processing, and in the afternoon - posters, demos and fast abstracts. The fast abstract session is now a trend in conference to enable people to have short reports about work in progress, not mature enough to have full papers. My fourth role in the conference will be to give a short talk about a work in progress we are doing in the area of geopsatial and spatiotemporal extensions for event processing language.
The last day (Thursday) will feature another keynote talk by Karsten Schwan, and three research sessions about variety of topics. I'll have to skip the last day, since I'll have to go for some meetings in the NY area on Thursday and Friday. I'll report more about DEBS 2009 later, looking forward to meet a lot of colleagues from the developing community.
Yesterday, Peter Niblett and myself, the co-authors of the EPIA book, had a meeting with the publisher and editor in order to summarize the review process of the first 1/3 of the book. One of the topics discussed was the ability of readers to have "hands-0n" experience.
As I explained in a previous posting, our approach is to teach our view about what is event processing and not teaching a specific language. We spent much of the review around this issue, the publisher suggested that for the benefits of readers who would like to have hands-on experience while reading the book we should provide a way to do it. What was agreed is that we'll use the use case that accompanies the book - the fast flower delivery --- to be a bridge between the book and the hands on experience. Since we would like to expose the reader to the multiple approaches that exists in the event processing area -- we'll provide the reader several alternatives to experiment, each alternative will be a language from different type -- it will be emphasized that the languages are just representatives, and the book does not endorse any of them in particular.
For each of the languages we shall provide:
- The fast flower delivery example coded in this language.
- Link to a site from which the reader will be able to download an implementation of this language (e.g. trial version of the product implementing this language) on which the reader will be able to run the example and experience in using that language.
Of course, this should be available to the readers without payment. In the next few days we shall talk with some of the language owners to select those who will be able to support these requirements.
I don't know how many readers will indeed spend time to take advantage of this option, but since 2 reviewers (out of 14) mentioned their wish to have "hands-on experience", I guess that there is certain segmentof the potential readers who will do.
Actually, I am a big fan of learning by hands on experience, my own experience from my long years as student is that courses in which I had substantial programming assignments are the course in which the knowledge gained in these courses is retained over time. I'll tell about my experience as a teacher on this issue another time. Stay tuned, this is going to be fun !
Today, again, I did not celebrate my birthday. I don't have a habit to celebrate a reminder to the fact that I am getting older. Interestingly, I got today much more (relative to recent years) birthday greetings in various communication ways -- E-Card through the Internet, phone calls, Email and even Real-time messaging. The reason that I got much more greetings relative to previous year can be attributed to the fact that I made many friends in the last year, but the more realistic reason is that certain social networks send people reminders about birthdays of other people in the network, which makes the knowledge about the birthday more accessible. Well -- none of my family remembered my birthday, here, at least, no change from previous years.
One of the many meetings I had last week was a teleconference with some IBM customers (I shall not expose their identity), my role in IBM is not in sales or sale support, but I am being called from time to time to participate in meetings with customers, by request of the people handling such meetings. The major issue that they wanted me to discuss is the benefit of using a middleware software for event processing (in the large sense) vs. hard coding it into the application code itself, like this customer is used to do.
Indeed, this is not a clear cut issue, there are two cases in which hard coding the event processing functionality make sense: either in the case that it is very simple, and thus it is not cost-effective to purchase, learn and deploy a specific product, or in the case that the functionality is too specific, and not covered by products, furthermore, it does not represent a ubiquitous requirement that makes it cost-effective for vendors to support it.
There are also some cases in which it is reasonable to write some functions in Assembly languages (even I did some of this in the past), but typically (most) people prefer to code in higher level language.
Likewise, in many applications, satisfy none of these conditions, and for these application is cost-effective to use generic software. The reason is that there are various common functions (e.g. filtering, routing, enrichment, transformation, pattern matching) that are repeating. Hard coding them means -- re-inventing wheels over and over again, instead of reusing existing implementations as "services", and enjoy other people's work that is being upgraded and optimized with time. This is similar to the reasoning of using other generic products -- messaging, workflows, databases, adapters, development environments and others.
The reaction of this particular customer was interesting: "what you are saying makes much sense, currently we are not used to think in terms of separating the event processing from the rest of the application logic, and we need to digest the idea". So, there will be a follow-up meeting to continue discuss it. I have seen this kind of reaction before, I think that a challenge of event processing vendors may be the competition with potential customers who do not understand the benefits above hard coding. But, this was also the case for other technologies that succeeded to cross this bridge. The growing number of event processing customers indicate that this thought is getting traction.
I think that this is also a topic for a community effort, that may be pursued by EPTS. More on this topic -- later.
Yesterday, I spent much of the evening in Nofim elementary school, my youngest daughter Daphna, is graduating from elementary school, and they have done their grand celebration, with a lot of speeches, and performance of the children in signing, dancing and playing. It was nice, but somewhat long (the speeches part), and they have done it outside in the hottest day of the year (so far).
Another experience is that I needed to replace the door lock cylinder in my former apartment, since she is going to rent it now. I know that the Americans tend to do everything (including oil change in cars) by themselves, when I joined IBM, I was surprised to discover that I am now a "technical person", actually my current IBM title is - IBM Senior Technical Staff Member; I have never thought about software development as a technical activity, in my mind, a technical person is a person that knows how to hold a screwdriver properly, a skill I never possessed, so I looked at the Internet and called a person whose profession is to do that, we set a meeting at 6PM, and I went out in the middle of a meeting to go there, at 6:10PM I called him, he said: the person is on his way, will be there within 10-15 minutes, I waited until 6:30, and my patient started to run out, I called again -- the answer: he is almost there, give him couple of minutes. I called again in 6:40, got the same answer. My last call was in 6:48, saying -- I am cancelling this order, and will try to find another one that when he says 6:00 he really means it. Today I called another guy from the Internet, and again set a meeting with him at 6:00PM, this time I called him before leaving home, and he said -- I'll be there in 30 minutes, and indeed, he was there 25 minutes later, and changed the lock. I told him that I am happy to see that there are some professional people who arrive on time, and he said that he as also a customer hates service people who are late... I always thought on punctuality as a value, unfortunately, I am (almost) the only one in my family who has a sense of time.
Anyway, I got today some spreadsheet that an analyst made to evaluate various event processing products, the good thing is that people are trying to devise such criteria, however, looking at the criteria there, my observation is that the people devised this list came with a long laundry list of criteria, many of them seem to me irrelevant for many of the applications I know, while others that I would imagine that will be there, are left out, or treated in a shallow way. This gets me back to the difficulty of devising performance benchmarks in this area.
My observation is that the event processing area is not monolithic; people are using event processing for different reasons, and have different functional and non-functional requirements and priorities. Trying to think in a monolithic way, may yield a result that is not valid for anybody. I think that there should be deeper research in this area and provide criteria for classes of applications; this classification is not easy, as the diversity of applications that each individual organization or vendor has, is limited. This is one of the things that require a community effort, and I hope that the EPTS use case working group will be able to provide it, so I am looking forward to hear their tutorial in DEBS 2009. This also relates to our goal of establishing event processing manifesto, which will be the main theme of the event processing Dagstuhl seminar in May 2010. Stay tuned for more.
I have written sometimes ago about creative use of Twitter as an event source; the recent days riots in Iran, where strict media censorship is imposed, indicate the role of means like Twitter. It turns out that Twitter has a major role in communicating to the outside world what is happening in Iran. The state department has asked Twitter to cancel scheduled maintenance to keep Twitter up and running all day. To all people who doubt the power of what can be expressed in 140 characters, it seems that Twitter is becoming a major event source in our universe, due to the ease of connecting it from everywhere. Of course, one can abuse any event source and send false information, but the mass quantity exposes the truth. I'll revisit text messages as event sources in later posts.
In the picture you can see manuscripts in some languages that require some expertise to parse...
Today I spent some time working on my part in the DEBS tutorial day entitled - Event Processing Language Tutorial, on behalf of the language analysis workgroup of EPTS. For this occasion we have also also launched an EPTS presentation logo, designed by Tammy Dekel, the graphical designer of my lab who also designed the EPTS logo.
The presentation starts with the following observation
and will include -- introduction including the language dimensions identified by the EPTS Language analysis workgroups, in-depth description of: SQL based languages, rule based languages, and EPA based languages, and then some advanced topics like: temporal properties, development environments and logic based semantics. The presentation is being prepared in a team work, and also be presented jointly by some of the members. We shall post the final version publicly, after the conference -- we need to finish it first.
There will be some more interesting tutorials in DEBS 2009 including the tutorial of the EPTS use case work group, a tutorial about event processing in capital markets, a tutorial about event processing in sensor networks and more. I am planned to be in Nashville for the DEBS conference, two weeks from now, and will report on the conference in this Blog. Stay tuned.
One of the authors' dilemmas about the right way to write ab event processing book is about - whether the book should concentrate around an "event processing language". One of the reviewers indeed suggested that the language will be centered about a specific executable language, and describe the concepts using the language, for the concepts that are not supported in the language, the reviewer proposed to include them in advanced topics section.
The idea is not new to us, when we discussed with the publisher about the book's structure, we considered such "bottom up" approach, however, we decided that it is methodically better to take "top down" approach. There are two reasons for it: the first -- we are building the book in a methodical way, and talking about the building blocks that are required to build applications, since different languages took different approaches and often the implementation blurs the concept, we thought the the bottom up approach will be a good way to teach a particular language but not the concept; the second -- the current languages follow various programming styles. We neither want to concentrate on a single programming style, nor to do a comparison, as this is not the goal of this book.
So - how are we approaching this issue:
- We have constructed a descriptive meta-language that follows the concepts we are defining. We may also provide some visualization of this language (checking this issue now), and enable to play with this language on the future book's website.
- We have asked all language owners (and some have responded positively) to provide their own implementations of the use case we are using in the book. We shall put code samples in all relevant chapters as provided to demonstrate the different attitudes. I'll write soon some comments on the use case, following some questions from language owners.
