I am now in my hotel room in Tarrytown, NY; one of the nearby towns is known as "Village of Sleepy Hollow", I always wondered about the origin of this name, and who wants to say -- I live in sleepy hollow.. I have seen some other strange name. For example, there is a USA small town called - "bird in hand".
I have noticed that Pedro Bizarro has posted the DEBS use case tutorial provided by the EPTS use case work groups on slideshare in form of 7 presentations. Enjoy!
It is interesting to note the "lessons learned" presentation, it summarizing ten lessons.
- Architecture
- Lesson 1: State-based event processing
- Lesson 2: Incident objects
- Lesson 3: Integration with other data management systems
- Lesson 4: System of systems
- Languages
- Lesson 5: Query languages
- Lesson 6: Classification and rules groups
- Lesson 7: Customization
- Glossary
- Lesson 8: Changes to glossary
- Use Cases
- Lesson 9: Changes to questionnaire
Lesson 10: Better instructions to describe a use case
I agree with the state-based lesson; I am advocating the notion of context which can include various types of state and segmentation, and I view it as a first class citizen in the model. This is both architecture and language issue.
What they call "incident object" seems to be an object that describes derived event that has some internal structure which can be updated by programs. This seems to be quite trivial notion that events need to have some structure, and be able to participate in processing other events, may be retained over time; I think that this is true for any event, as event may also mean "event object". About the question whether it can be updated by programs, this is a matter of taste, since they want to keep these objects in temporal databases, temporal databases has adopted an "append only" model, where changes create other objects, and there is some reasoning behind it. I may dedicate a postings to this topic.
Integration with other data management systems -- given that Dieter Gawlick is co-leading this workgroup, one can expect that there is a database-centric view of event processing, his views on this topics are well known. I agree that databases can be integrated with event processing systems for various reasons (state keeping, reference, meta-data holding, high availability...), but event processing may also be integrated with a messaging system, with business process management system, with a BAM system, and with sensor networks, so I would extend it to integration with other parts of the enterprise architecture.
System of systems has to do with modularization and some implementation issue (e.g. bounds of local and global states). In my view EPN can be defined recursively such that an EPA can represent an EPN, thus we can express "system of systems". I am not sure that this requires any conceptual difference.
Query languages -- I am not sure I got the exact point here, but it seems that all event processing styles require to express conditions such as predicates for various uses. I think this is already happening. The issue whether internal state can be queried is different, and not relate to the language issue.
Classification / customization-- This is again one of the context dimensions, as context can classify populations according to classification predicates. Languages that are more syntactic oriented like the SQL derivatives hide classification inside SQL Where, but where context is supported as a primitive concept then classification is an explicit construct; this also allows customization. The use of vocabulary, as been used in rule languages, can be helpful here.
Glossary -- I'll leave the glossary discussion to the glossary workgroup for a while.
Use cases description --- the goal of this entire exercise was to "debug" the use cases template; I hope that the use case workgroup will finalize the template soon, and move to the next phase -- getting a substantial number of using cases and draw conclusions about their types, requirements etc--- it will be a great help to the community if we'll succeed to classify the applications to a finite, small, number of classes, where in each of them we'll be able to specify important functional and non-functional requirements. More on this -later.
Tarrytown, NY. I have left Nashville this morning and arrived to Westchester county, NY, to do some IBM meetings on the way back from DEBS. I think that the nicest analogy I got from the conference came from Alex Buchmann keynote address, he used some other pictures, but provided the analogy that while regular programming is like drinking with a straw, this is good when the data is standing, while the data is moving, like in event processing, using the same kind of thinking is similar to use a straw to drink from a waterfall. I like this example. It requires some education to understand that event driven is a different paradigm of thinking, and this impact the way we should think about the programming model as well as the execution model of these types of applications. This also links to the discussion on education that will create a critical mass of developers who understand this paradigm. It is somewhat different, but it is not rocket science, with some education, I believe that it is doable.
I'll end with some travel anecdotes: Today I rented a car in the airport, got to the car and did not find a car key, just a remote control; it took me two minutes to understand that the remote control serves also as a car key, there is a hole in which it has to be inserted to, and when it is there, there is a on/off button which turns the engine on or off. I showed it to one of the people I've talked with in the place I visited and he told me that he has even more sophisticated electronic key that he does not need to insert it anywhere, it is enough that it is in his wallet that the car identifies it and enables to turn on the engine.
Another anecdote that always puzzles me is hotels' superstitions regarding the number 13. My Nashville hotel was semi superstitious, it did not have 13th floor, jumped from 12 to 14, however I have stayed in room 413. Now I am in a different hotel, this time in room 412. For curiosity I have looked for room 413, and did not find it, it seems that the level of superstition here is higher... I never understood this superstition, but this is not the only thing that I don't understand... Go figure.
I am still in the DEBS conference venue, in a time gap between the conference sessions and the conference dinner that will be held here in an hour or so. This is the time to provide some more comments about DEBS 2009. The conference will continue tomorrow, but I'll skip the last day, going to NY for some IBM meetings. Spending whole three days in a conference is a heavy toll on my busy schedule, but it is fun. The next conference will be held in Cambridge, UK, and the call for papers (above) has been distributed. Today Alex Buchmann gave the second keynote address about some event processing applications and research challenges, I hope that Alex will make his slides available to the public. Also today there have been sessions of demos and posters, mostly by Ph.D. students, mostly concentrate on the transport area, pub/sub, messaging etc... We need more Ph.D. students on the event processing topics. Next there has been a fast abstract session that present work in progress in 10 minutes. I have presented a work in progress on spatial aspects of event processing, and will write more in the future about this activity.
Back to yesterday, the industry track was divided into two parts: industrial reports and a panel. In the industrial reports there was one, presented by Florian Springer, that the audience will remember for long time, not so much because the content of the talk, but since Florian made a point about importance of standards, by showing real-life examples for standards, and the example he chose was condoms, which, according to Florian, have a standard - they are all in the same size. People kept talking about it.
I have moderated the panel about academia/industry relations in event processing, and posed the following questions (see slide below)
The first question was, whether the academia should work on incremental stuff related to current technology, or on disruptive technologies, making current technologies obsolete, which was translated to the question --- do we want to clone John Bates to develop new technologies that will make the technology created by the original John Bates as obsolete. John Bates, who was in the audience, reacted by saying that his wife will not like the idea of cloning him, since she thinks that one of this type is more than enough. The panelists were two persons from academia, and two from industry. One of the industrial people was Richard Tibbetts, Streambase CTO, who showed the famous Gartner hype-cycle picture, claiming that the event processing area is already approaching the plateau. I think that he is a bit optimistic, in my opinion, we are not even at the peak of inflated expectations... 
There was also some discussion about teaching event processing courses, and the fact that today it takes a lot of time to get a university graduate being effective in developing event processing applications, since they need to be taught to think in a different way. There have been some more talks, but I am tired from writing, so that's all for now.... Tomorrow early morning -- flying to NYC.
Yesterday evening the "open business meeting" of DEBS took place. This meeting is intended to draw lessons from the organizers of this year's conference, and obtain decisions about the next years' conferences. I participated in the business meeting two years ago in Toronto, where Mani Chandy and myself have challenged the DEBS organizers to extend the scope of the conference, and this seems to happen, the conference is shifting its focus from being mainly a "pub/sub" conference, to a conference that covers the entire aspects of event processing (I view pub/sub as an aspect). Last year in Rome, we suggested to host DEBS 2010 in Haifa, however, the decision was to do it in UK, and give us a chance to host DEBS 2011. Indeed, the next DEBS will take place in Cambridge, UK. In the meeting yesterday the decision about DEBS 2011 organization was to be decided. A day before, Arno Jacobsen, who is the person behind the DEBS conferences (chair of DEBS steering committee) addressed me and said that they still want me to organize DEBS 2011, but in the New York area instead of Haifa. He mentioned two reasons - the first one is that they want to rotate the conference between Europe and North-America, so in 2011, it should be held in North-America again, and the other reason is that people may be afraid to come to Israel, which will reduce the number of participants. This is an issue, since in Nashville we have a substantially less participants than we had last year in Rome, and they thought that NY is easily accessible place both to Americans and Europeans. I think that the reason that there are less participants this time is a combination of the economic situation, which travel restrictions in many places (especially in industry), and the location, while the conference is in a nice setting, it needs connection flights even for some Americans. Another reason is that I think that most of the community in Europe is larger than the north american community, also a couple of participants had to cancel due to visa issues. We don't know what will be the economic atmosphere in 2011, but hope it will be changed. About the fact that the potential to have more participants in NY than in Haifa, I tend to agree. About the argument that people may be afraid to come to Israel, I stated my opinion publicly in the business meeting yesterday that this is a distorted perception that is a result of hostile media coverage in the western world. Personally I feel safer in Haifa than in Nashville and certainly safer than in London, New York, or Paris. Having said that, I am aware of the fact, that it is difficult to fight perceptions, see my posting (on another matter) about historic truth vs. archaeological truth
Bottom line -- together with my colleagues in IBM Watson Research Center, I agreed to organize DEBS 2o11, this will take place in one of the IBM sites in the NY areas, we still need to determine the exact location. Stay tuned.... Meanwhile, we still have to finish the DEBS 2009 and survive the next 2 years...
Stratification is one of the terms that computer scientists borrowed from Geology. In 2007, Ayelet Biger, my former M.Sc. student has done her thesis about Complex Event Processing Scalability by Partition which looked at semantic partition of an EPN graph to strata, where in each stratum all agents are independent and can run safely in parallel. Today we have reported about 2008 research project that took the stratification idea and developed a system to assign agents to machines in a distributed environment. Geetika Lakshmanan delivered the talk about this project today. I have posted it on Slidshare. Enjoy !
This project is an interesting example for a life-cycle of a project:
- It started as an academic thesis;
- It has flown to a research project within IBM Haifa Research Lab ;
- After showing promising results in the lab, it evolved to a more "down to earth" project that deals with assignment of agents to machines and threads within the IBM product - WBE-XS (Websphere Business Events - Extreme Scale) which enable event processing on grid environment. This project has to take into account products and their implementation, deal with product instrumentation, and other stuff that pure research projects do not deal with.
While starting an idea in the academia and then going to a start-up has its magic, the work in IBM Research enables to get stuff from academic projects until impact through products, and using my hat as an adjunct professor in the Technion, this is possible. However -- as noted before, getting things pushed in a big company are not necessarily easy.
In my role as the industrial track chair of DEBS 2009, I have been asked to invite an industrial keynote speaker, and this year I chose John Bates, the General Manager of the Apama Division of Progress Software, and a person that made a transition from being a faculty member in Cambridge University, leading one of the first start-ups in this area, and now managing a division in a larger company. This was certainly a good choice; John outlined the assumptions he had as a researcher, and the confrontation between these assumptions and the reality that forced them to find a low hanging fruit, until the bigger market will be mature enough for the ideas for event processing, and they found trading in capital marketing as this low hanging fruit, and the early adopters of such technologies. However, the maturity in thinking about this area now takes it to more areas. As for John's prediction for what will happen in the event processing market in the next five years, he mentioned the following points:
- Event Processing will be used for tracking everything -- car, plane, bag, package, ship
- EDA -- will achieve wider adoption -- federating services in the enterprise, providing agile enterprise nervous system; The next wave of SOA.
- Event processing everywhere -- EP in the cloud.
- Event Processing will be part of bigger event-driven business process management market - BPM, rules, event processing are going to merge.
- Event processing will be embedded in many vertical applications.
More about further sessions in DEBS 2009 - later
This is the Vanderbilt University Law School, where the DEBS conference is being held this year. Today I spent the day in a U-shaped seminar room, with a lot of ex-Deans of the school hanging on the wall. The first day has been the tutorials day. In the first half of the day I attended the "use cases" tutorial in which the EPTS use cases workgroup, chaired by Pedro Bizarro and Dieter Gawlick, who, together with some more members of this workgroup presented some use cases, together with lessons learned. Paul Vincent has written a review (with some criticism) about that tutorial. I wonder how many bearded academics he counted in the conference -- indeed, once in academic conferences one could see all the prominent scientists carrying a beard, but today, most of them are shaving their entire face... Dieter who presented some of this tutorial has a beard, but he is from Oracle and not from the academia. Anyway, the idea to have an EPTS award (donated by TIBCO) to the application of a year is a good one, I'll bring it to the EPTS steering committee. We also discussed (after the end of the program) topics related to the use cases workgroup and relations with other EPTS workgroups.
In the second half of the day, Jon Riecke, Adrian Paschke and myself have presented the "event processing languages" tutorial. I am not reviewing my own talks, so instead -- you can look at the slides and review it yourself. More about DEBS 2009 -- tomorrow.
Early morning in Nashville. I arrived here yesterday, and after some rest, went to dinner with some of the people in the community (this was initiated by Matthew Cooper from Eventzero who invited some people). In the dinner I was sitting next to Dieter Gawlick from Oracle, one of the most active people in the community. Dieter is recently advocating the concept of "state based event processing", based on some work he is doing. Today I'll hear more about it in the use cases tutorial, and then in a later meeting with Dieter to get feedback from the use cases workgroup to the language analysis and reference architecture work groups. My initial impression is that in the term "state" he more or less means what we call "context". I have just written the first draft of the chapter in EPIA book defining and describing the term context (will be available on the Web within a few weeks I hope). It will be interesting to see if how the notion of context will further evolve. Most event processing languages have very modest support in context these days (e.g. time window as a simple case of temporal context). I'll write again about contexts and states in one of the next postings. Today -- the DEBS conference start, and the tutorial day (last year it was the most interesting day of the conference, let's see if we'll keep the standard high). Now -- going over my part in the event processing language tutorial, so I'll not mix up the English words...
I am now in my hotel room in Tarrytown, NY; one of the nearby towns is known as "Village of Sleepy Hollow", I always wondered about the origin of this name, and who wants to say -- I live in sleepy hollow.. I have seen some other strange name. For example, there is a USA small town called - "bird in hand".
