Friday, July 3, 2009

On DEBS 2009 - next week

Tomorrow night I am planned to start the long way to Nashville, to attend the DEBS 2009 conference. DEBS is originally a conference founded by the distributed computing guys, what is known as the "pub/sub" community. Two years ago, Mani Chandy and myself came to the DEBS steering committee and challenged them to extend the conference such as it will be a general event processing conference, and work with them to make it the scientific "flagship" conference of event processing. Looking at the DEBS technical program, the first day (Monday) is the tutorials day. It will have six tutorials, five of them are about event processing, and two out of the five were created by EPTS work groups: the use case work and the language analysis work group. Giving part of this tutorial (the other parts will be given by Jon Riecke and Adrian Paschke) will be my first role in this conference. We shall make these tutorials available to the public after the presentation. The second day (Tuesday) will start with a keynote address by John Bates, Apama GM, and one of the pioneers of the event processing area. He will be the industrial keynote speakers, and as chair of the industry track, I will have the honor of introducing him, my second role in this conference. The first research session on distributed event processing will have a paper which I co-authored on stratified approach for supporting high throughput event processing applications, however, I will leave the presentation to the younger generation (Geetika Lakshmanan from IBM Watson Research Center will give this talk). The second research session will be about pub/sub, but one of the talks has the event processing phrase in the name. Then we'll have the industrial session, and my third role will be to chair it. This session will consist of four industry report, and industrial panel about academic-industry collaboration in event processing. We plan to have four panelists, two from academia, one customer, and one vendor, and discussion with participation of the audience. DEBS originally was almost purely academic conference, but we have injected some the industrial participation, and believe this partnership is important for both sides, and more important, for the future of the event processing area. In the DEBS business meeting, later that evening, I plan to propose hosting DEBS 2011 in Israel. The third day, Wednesday, will have a keynote talk of Alex Buchmann, an old colleague from the active database era, who is one of the DEBS founders. The research session of that day will deal with complex event processing, and in the afternoon - posters, demos and fast abstracts. The fast abstract session is now a trend in conference to enable people to have short reports about work in progress, not mature enough to have full papers. My fourth role in the conference will be to give a short talk about a work in progress we are doing in the area of geopsatial and spatiotemporal extensions for event processing language.
The last day (Thursday) will feature another keynote talk by Karsten Schwan, and three research sessions about variety of topics. I'll have to skip the last day, since I'll have to go for some meetings in the NY area on Thursday and Friday. I'll report more about DEBS 2009 later, looking forward to meet a lot of colleagues from the developing community.

Wednesday, July 1, 2009

On EPIA and hands-on experience

Yesterday, Peter Niblett and myself, the co-authors of the EPIA book, had a meeting with the publisher and editor in order to summarize the review process of the first 1/3 of the book. One of the topics discussed was the ability of readers to have "hands-0n" experience.

As I explained in a previous posting, our approach is to teach our view about what is event processing and not teaching a specific language. We spent much of the review around this issue, the publisher suggested that for the benefits of readers who would like to have hands-on experience while reading the book we should provide a way to do it. What was agreed is that we'll use the use case that accompanies the book - the fast flower delivery --- to be a bridge between the book and the hands on experience. Since we would like to expose the reader to the multiple approaches that exists in the event processing area -- we'll provide the reader several alternatives to experiment, each alternative will be a language from different type -- it will be emphasized that the languages are just representatives, and the book does not endorse any of them in particular.

For each of the languages we shall provide:

  • The fast flower delivery example coded in this language.
  • Link to a site from which the reader will be able to download an implementation of this language (e.g. trial version of the product implementing this language) on which the reader will be able to run the example and experience in using that language.
Of course, this should be available to the readers without payment. In the next few days we shall talk with some of the language owners to select those who will be able to support these requirements.

I don't know how many readers will indeed spend time to take advantage of this option, but since 2 reviewers (out of 14) mentioned their wish to have "hands-on experience", I guess that there is certain segmentof the potential readers who will do.

Actually, I am a big fan of learning by hands on experience, my own experience from my long years as student is that courses in which I had substantial programming assignments are the course in which the knowledge gained in these courses is retained over time. I'll tell about my experience as a teacher on this issue another time. Stay tuned, this is going to be fun !

Sunday, June 28, 2009

On hard coding event processing functionality

Today, again, I did not celebrate my birthday. I don't have a habit to celebrate a reminder to the fact that I am getting older. Interestingly, I got today much more (relative to recent years) birthday greetings in various communication ways -- E-Card through the Internet, phone calls, Email and even Real-time messaging. The reason that I got much more greetings relative to previous year can be attributed to the fact that I made many friends in the last year, but the more realistic reason is that certain social networks send people reminders about birthdays of other people in the network, which makes the knowledge about the birthday more accessible. Well -- none of my family remembered my birthday, here, at least, no change from previous years.

One of the many meetings I had last week was a teleconference with some IBM customers (I shall not expose their identity), my role in IBM is not in sales or sale support, but I am being called from time to time to participate in meetings with customers, by request of the people handling such meetings. The major issue that they wanted me to discuss is the benefit of using a middleware software for event processing (in the large sense) vs. hard coding it into the application code itself, like this customer is used to do.

Indeed, this is not a clear cut issue, there are two cases in which hard coding the event processing functionality make sense: either in the case that it is very simple, and thus it is not cost-effective to purchase, learn and deploy a specific product, or in the case that the functionality is too specific, and not covered by products, furthermore, it does not represent a ubiquitous requirement that makes it cost-effective for vendors to support it.

There are also some cases in which it is reasonable to write some functions in Assembly languages (even I did some of this in the past), but typically (most) people prefer to code in higher level language.

Likewise, in many applications, satisfy none of these conditions, and for these application is cost-effective to use generic software. The reason is that there are various common functions (e.g. filtering, routing, enrichment, transformation, pattern matching) that are repeating. Hard coding them means -- re-inventing wheels over and over again, instead of reusing existing implementations as "services", and enjoy other people's work that is being upgraded and optimized with time. This is similar to the reasoning of using other generic products -- messaging, workflows, databases, adapters, development environments and others.

The reaction of this particular customer was interesting: "what you are saying makes much sense, currently we are not used to think in terms of separating the event processing from the rest of the application logic, and we need to digest the idea". So, there will be a follow-up meeting to continue discuss it. I have seen this kind of reaction before, I think that a challenge of event processing vendors may be the competition with potential customers who do not understand the benefits above hard coding. But, this was also the case for other technologies that succeeded to cross this bridge. The growing number of event processing customers indicate that this thought is getting traction.

I think that this is also a topic for a community effort, that may be pursued by EPTS. More on this topic -- later.