The event processing manifesto, chapter 1, deals with the question - why event processing should be used.
The idea is a service to the community to educate and help resolve misconceptions and confusions that exists among some people on the role and usefulness of event processing approach and platforms.
The slide above is a table copied from the document (table 1.1) which sets the main decision criteria:
- Event driven nature
- Events rate
- Application complexity
- Timeliness
In the table there are various combinations and recommendations.
The event driven nature is, of course, a major factor. Applications with medium and high event driven nature will naturally gain more than applications with low event driven content.
The event rate is another factor - although it determines more of the non functional requirements than the decision whether event processing should be used. However, the higher the event rate is, the higher the benefit of using optimized framework.
The application complexity has various dimensions - I will discuss application complexity in a separate posting, but as seen in the previous posting, the higher the complexity is, there is more benefit to use event processing platforms.
Last but not least is timeliness - if timeliness is high, then again there is a benefit of using event processing platforms, if timeliness is low and complexity is high, it depends on what exactly are the complexity characteristics of the specific applications.
More on application complexity and no the manifesto - later.
This is a map of Finland, the location of our family vacation for this year. The vacation is planned to start in Saturday, and I'll be disconnected from the cyberspace for 15 days. Working late at night to advance in the EPIA book we are writing.
It seems that this is the time of the year of analysts report, the community blogland was full of references to the Forrester wave report - Complex event processing platform, Q3 2009, dated August 4, 2009.
I will not comment about the grades that they gave the different products, the reason for that is that I decided that in my role as chair of EPTS, I see my role to work on the coop side of the coopetition, and leave the competition side of the coopetition to others. I think that the main competition is not between the different vendors (though they have point competitions), but against the barriers that the event processing area has to fulfill its potential and become a pervasive main-stream technology.
The Forrester reports starts the executive summary by saying: "Forrester evaluated nine complex event processing (CEP) platforms using 114 criteria".
Without getting into the long list of criteria (not part of the report itself, but I managed to look at it), I have some doubts about the ability to get a meaningful information to customers by weighing 114 criteria. There are two reasons, one is practical and one is methodological.
On the methodology side, the compensatory model of decision making advocates weighing of many criteria, however, experience shows that the actual decision making model is lexicographic, meaning ordering the criteria according to importance, and making the decisions according to the most important criteria. People may use a compensatory model of weighing a lot of criteria, if their organization require them to work this way, but this is done only as justification to decision that has already been made by the lexicographic model.
Let's move from decision making theory to the event processing universe. The event processing universe is diversified from both functional and non-functional requirements point of view. I really don't believe in a "one size fits all" in anything related to this area, and this goes also for set of evaluation criteria. Getting criteria that are good in variety of cases and weighing them together may not get a good solution to any particular case. The more practical approach is to set a collection of relatively small sets of important criteria, and also segment the space of application, and assign a set of criteria to each segment. Anybody that will manage to do it, will help customers more to make the best decision for a particular case. I hope that EPTS, through its use cases workgroup will be able to provide this segmentation, and this will be the starting point for analysts to come with the right criteria for each segment.
Yesterday, I spent much of the evening in Nofim elementary school, my youngest daughter Daphna, is graduating from elementary school, and they have done their grand celebration, with a lot of speeches, and performance of the children in signing, dancing and playing. It was nice, but somewhat long (the speeches part), and they have done it outside in the hottest day of the year (so far).
Another experience is that I needed to replace the door lock cylinder in my former apartment, since she is going to rent it now. I know that the Americans tend to do everything (including oil change in cars) by themselves, when I joined IBM, I was surprised to discover that I am now a "technical person", actually my current IBM title is - IBM Senior Technical Staff Member; I have never thought about software development as a technical activity, in my mind, a technical person is a person that knows how to hold a screwdriver properly, a skill I never possessed, so I looked at the Internet and called a person whose profession is to do that, we set a meeting at 6PM, and I went out in the middle of a meeting to go there, at 6:10PM I called him, he said: the person is on his way, will be there within 10-15 minutes, I waited until 6:30, and my patient started to run out, I called again -- the answer: he is almost there, give him couple of minutes. I called again in 6:40, got the same answer. My last call was in 6:48, saying -- I am cancelling this order, and will try to find another one that when he says 6:00 he really means it. Today I called another guy from the Internet, and again set a meeting with him at 6:00PM, this time I called him before leaving home, and he said -- I'll be there in 30 minutes, and indeed, he was there 25 minutes later, and changed the lock. I told him that I am happy to see that there are some professional people who arrive on time, and he said that he as also a customer hates service people who are late... I always thought on punctuality as a value, unfortunately, I am (almost) the only one in my family who has a sense of time.
Anyway, I got today some spreadsheet that an analyst made to evaluate various event processing products, the good thing is that people are trying to devise such criteria, however, looking at the criteria there, my observation is that the people devised this list came with a long laundry list of criteria, many of them seem to me irrelevant for many of the applications I know, while others that I would imagine that will be there, are left out, or treated in a shallow way. This gets me back to the difficulty of devising performance benchmarks in this area.
My observation is that the event processing area is not monolithic; people are using event processing for different reasons, and have different functional and non-functional requirements and priorities. Trying to think in a monolithic way, may yield a result that is not valid for anybody. I think that there should be deeper research in this area and provide criteria for classes of applications; this classification is not easy, as the diversity of applications that each individual organization or vendor has, is limited. This is one of the things that require a community effort, and I hope that the EPTS use case working group will be able to provide it, so I am looking forward to hear their tutorial in DEBS 2009. This also relates to our goal of establishing event processing manifesto, which will be the main theme of the event processing Dagstuhl seminar in May 2010. Stay tuned for more.
Typically, I refrain from reacting in this Blog to any marketing material presented by vendors, a restriction I have taken upon myself as the chair of EPTS. I am not deviating from this rule, but since my friends in Coral8 have posted their article entitled: Comprehensive Guide to Evaluating Event Stream Processing Engines on David Luckham's site, as a vendor-neutral service to the community, I am taking a freedom to put some footnotes to this paper.
On the positive side, I think that this type of work is useful, and discussions about it is also useful, and many of the criteria presented are valid. We in IBM have devised in the past criteria for evaluation for internal purposes that included many of the mentioned criteria, I have to check if we can expose them.
On the critic side - here are several comments:
1. The first claim is that the authors view "event stream processing" and "complex event processing" as one and the same, saying that customers do not make distinction between terms, and saying that there is no agreed upon terminology. I am referring the authors to the EPTS glossary as a reference for terminology. But regardless of that, I would agree that customers typically don't care what TLA is used, the substance is more important.
2. Giving the statement that the coverage of this document is ESP and CEP which are one of the same, have created the feeling that this document is general, however, reading further I find out among the criteria that define what is ESP engine the following condition: "...process large volumes of incoming messages or events". This criterion confuses me -- is that a fundamental property of ESP/CEP engine -- I have heard in the recent year some analysts talks saying that actually most of the potential EP applications are not the "high volumes" ones, furthermore, the customers I know have various degrees of event volumes, some of them high, some low -- so maybe this is not part of the definition of what is an engine, but an evaluation criterion for certain amount of applications.
3. Reading further I see terms like: continuous queries, windows -- terms that already assume a certain type of implementation (indeed --- query-based stream processing), this fits the title of "event stream processing" assuming that there is an agreement that this is what ESP is, however, it does not represent the entire spectrum. Continuous queries is a technique that is intended to achieve some functionality, that can be achieved in other means.....
Personally I believe that "one size fits all" does not work, and that different event processing applications have different functional and non-functional requirements. There are applications in which various performance aspects are more or less important, note that there is also no standard benchmarks yet. I hope that the work of the EPTS work group on use cases that is planned to result in classification of event processing applications will result in a finite, manageable number of application classes, so the evaluation criteria can be partitioned by type.
And -- if possible, hands on experience indeed makes the evaluation more accurate and removes noise of preconceptions and false assumptions... More on evaluation - later.
