The various chapters of the event processing manifesto, published yesterday, hold some interesting insights about the state of the practice and the future of the event processing area. Chapter 1 written by a team lead by Robert Berry has provided some insights and directions about why use event processing systems. The figure below taken from the manifesto (figure 1.5, page 20) is talking about a frequently asked question - whether an event processing computerized system should be used at all, and if yes - should it be implemented as part of the conventional programming or using a dedicated COTS event processing system.
As seen in this figure - this is a function of complexity. There are various complexity factors, defined earlier in the document:
1. Degree to which the application is expected to change over time, e.g., with new
event sources, new interactions and new responses expected
2. Numbers and types of event sources
3. Numbers of consumers of information communicated in the events
4. State and context management
5. Opportunity to create new value, e.g., by introducing reflection and introspection
Back to the decision - in some cases the complexity is low, and there are no really processing required except for getting the events and send them to some person directly or displaying them on a dashboard.
In this case the most cost effective solution is "manual", namely, a human is the event processor and no computerized system is required to do any processing, except for routing, which can be done by any messaging or DBMS system.
The first vertical bar (from left) is the complexity break-even point between the "manual" approach and the "build" approach, beyond this bar it is cost-effective to construct an event processing computerized solution, but the complexity of such system is quite low, typically doing filtering, some transformations, maybe simple aggregations, typically without strict timing constraints. In such cases learning and using a COTS system might be an overkill, and it is relatively simple to develop as part of regular programming.
The second vertical bar is the complexity break-even point between the "build" and "buy", beyond this bar, it becomes cost-effective to invest in COTS (the right one which satisfies the requirements, of course).
Note that sometimes the decision should be forward looking, if there is a plan or prediction that the complexity of the application will increase within the short and medium range, then it should also serve as consideration to avoid re-writing of the system within a relatively short period of time.
I think that we need some empirical research to determine how to measure these two break-even points in a more exact way.
Our next mission is probably to devise best practices for the community on this and other issues, and it is already proposed to have it as one of the next working items for EPTS.
Today, the Dagstuhl publication service published the "Event Processing Manifesto" which is the result of the work started in the Event Processing Dagstuhl seminar that took place in May 2010 (after long pregnancy). This 60 pages document is a seminal work that is a result of the collaborative work of the 45 participants seen in the picture below, and quite a lot of editorial work that I have done with the help of Sharon Geva and Chani Sacharen, from our local technical editing department. The document has six chapters (five original workgroups, one of them decided to generate two chapters). The chapters deal with: why use event processing, what is the functionality of event processing, how is event processing related to other disciplines, what is the current related standards and what should be the future roadmap, what is the grand challenge in the entire industry level, and what are the shorter term research issues that will promote the area. I'll dedicate a posting to each of these chapters in the next few days. The EPTS virtual symposium, planned for March 24, will provide an opportunity to get perspectives on each chapter from the person who lead the construction of this chapter.
This is the revised logo of DEBS 2011, the conference's webmaster, Darko Anicic, noticed that the dates are wrong (when designing the logo we did not take into consideration the PhD workshop which is part of the conference).
Anyway -- the program chairs announced today on a one week extension, due to some requests.
The new deadlines are: March 7 for abstracts and March 14 for full submissions.
These apply for the research, industry and tutorial tracks.
This will give opportunity to those who still hesitate to submit!
The news of this week which I got in multiple copies from multiple sources is the launch of a product called SAP BusinessObjects Event Insight, the SAP website that describes this product exhibits the picture I have copied here, probably a business person (by the dress) that get some event insights by phone, and looks happy about these insights. The product is probably a descendant of Aleri, which purchased Coral8, and later sold its assets to Sybase, which in turn was purchased by SAP. It is interesting to note that it is branded as "BusinesObjects" which is another SAP acquisition of a French BI company. This shows that SAP positions event processing as part of BI suite, which follows one of the current trends. When we started the event processing community meetings in 2006 one of the participants said that event processing will hit the mainstream of computing when all the four big software companies -- IBM, Oracle, Microsoft and SAP will have products in this area. With this product launch, SAP joins the three others who are already there, along with some of the medium size software companies Progress and TIBCO. While all the "big dogs" are there, I think that there is also a place in the ecosystem to small companies and startups that will go for either niche requirements or domain specific products. It is also conceivable that small companies will create disruptive technologies that will take us to further generations -- meanwhile, greetings to those behind the project, hope to be able to learn more about it.
One of the amazing achievements of IBM Research is the recent triumph of the "Watson" program, whose avatar is seen above in the Jeopardy! game, beating two human champions. This raises back the question whether computer thinks, and whether we are getting closer to the "singularity" vision between human and machine intelligence. While this game is a question answering one, a complete machine intelligence will also require the ability to detect events and contexts, and react within a changing environment. I suggested (as a joke) that the next game that a humanoid robot should strive to is the "survivor", which requires other capabilities, like social capabilities, but I think that this challenge is somewhat beyond even the current wildest dreams.
Have not written for the last week, spent some of it in the IBM La Gaude site, near Nice, in South France, the La Gaude site is on mountain and can be seen in a bird eye's view here. The meeting was an internal meeting of "technical leaders" (whatever it is) in IBM South and West Europe, and was kind of educational and networking session. Back home - occupied by some family matters, so now I found some time to catch up. First looking at the Blogland, one can find two old bloggers returning after long silence, one of them is Brian Connell, who returned to blogging after a long time, the other is Marco from Rulecore, who also returned to blogging after a long silence. In a comment to my Blog, the Rulecore guys are wondering why Rulecore is not mentioned in the ACM Computing Survey upcoming survey on event processing while other are mentioned. I think that the reason is quite obvious, while I am familiar with Rulecore (they participated in the implementation of the EPIA example; my students used them in project several times), if one wants to be known by the academic community, then one has to participate in the research oriented conferences, publish (at least industrial) papers etc --- in our world, visibility is the name of the game, so you might consider sending an industrial paper or experience report to DEBS 2011, or participate in the DEBS 2011 challenge.
Another blog (Jason Irwin's) that I don't typically follow provides a new review of the EPIA book.
My next trip is planned for late March in the USA, I was invited to give a tutorial on event processing as one of the highlights of the OMG Technical Conference, I might also do some other meetings in the USA (customers' meeting, IBM meetings, university visits), but still working on the plan.
Paul Vincent has recently blogged about event driven applications within the applications types cited by the Wired magazine in an article talking about further AI applications. There are other work within the AI community which look at the short and long term impact of AI research on the universe, such as the AAAI presidential panel on AI research. There is a lot of potential in collaboration between the AI and EP communities, from the EP side, there are variety of AI techniques that can be used, and from the AI perspective it provides cases in which AI techniques can be specialized in order to be used in practice.
Recently my colleague Yagil Engel , an AI person we recruited to investigate applications of AI techniques for EP, and myself have been notified that our tutorial proposal for AAAI 2011, a top AI conference, was accepted.
So according to the plan we'll be talking four hours in August about the relationships between EP and AI, the first two hours are planned to provide a crash course on event processing for the AI audience, while the second half is planned to discuss various EP challenges that relate to various AI areas (planning, learning, uncertainty handling, autonomous systems and more. The idea is to establish a dialog with more people in the AI community and pursue this line of research further. The down side: we actually need to prepare now a four hours tutorial. Will write more about it when it is finished (I am sharing all my public presentations on slideshare).
