This is a picture of Akko (AKA Acre), an ancient city with walls, middle eastern typical market with the smells of spices, an fisherman's harbor. I am hosting now some visitors from Germany, Rainer von Ammon and his colleagues from CITT to discuss some collaboration topics including a consortium for an EU project that we are establishing with other partners. Unfortunately they chose to arrive in the most rainy period we have this year, so we could not do much sightseeing today, however, we succeeded to get two hours break in the rain to stroll around the old city of Akko.
I'll get back to the discussion about the questions I posed yesterday soon, as I would like to see if more people want to react before stating my opinion (I am in learning mode...).
Today I'll write something about Information Fusion and its relationship to event processing; I came across a recent survey article in ACM computing surveys about data fusion.
There are various kind of fusions - data fusion, information fusion and sensor fusion -- and all of them are intended to get information from distinct sources blend it together and understand what has happened. A very simple example of sensor fusion is in traffic monitoring, there is a sensor that senses the speed of a car, there is a camera that takes pictures of the car and its license plate, fusion of both can identify the fact that a certain car has violated the speed laws, this is a relatively simple case that requires some basic image processing, but it is quite easy to determine what happened. This is, of course, very simple case, and in the area of military intelligence it is much more complicated to understand what happened / happening / going to happen and some techniques are being used. The Center for Multi source information fusion in University of Buffalo maintains a site with collection of bibliography about fusion issues including tutorials and their proposals to modify the relatively old JDL model, so you can find much more information there.
So where is the confusion ? --- there are people who confuse event processing with some other different areas, somebody in IBM who saw an illustration of event processing network once tried to convince me that we are re-inventing workflows, some data management people think that event processing is just a footnote to existing query processing, everyone with a hammer looks at the rest of the world as a bunch of nails; Likewise, there are people who confuse fusion with event processing.
So what is the infusion? the fact of the matter is that information fusion and event processing are complementary technologies. The goal of fusion is to determine what happened, i.e. to determine what is the event that has occurred. Event processing is processing the event after somebody determined that the event happened it has multiple goals, the techniques are different, fusion is using conflict resolution techniques and stochastic modeling, event processing is using pattern matching, transformation, aggregation etc. Thus an event can be created using fusion techniques and then processed using an event processing system -- this is the infusion.
However -- there is also a potential synergies between these two applications - a partnership of fusion technology as a preprocessor for events and event processing can be beneficial for certain applications, this is the most obvious synergy. Another type of synergy is that techniques used in fusion can be used in event processing and vice versa, this is an interesting direction to investigate further and also investigate possible real applications for it. More on this - later.
I am sitting now in my living room, watching the heavy rain outside.Rainy; We did not get a lot of rain this winter, however when the rain comes it tends to come in the wrong times, like weekends (Friday is part of our weekend which is Friday and Saturday, Sunday is a normal work day, it is used to catch up on things since in this day our colleagues from abroad are idle and there are no conference call and other interactions).
Today I would like to concentrate on the question --- what level in the Enterprise is event processing for, I had a recent discussion with somebody who investigated the BRMS market and asked him this question about BRMS, and the answer was the mostly BRMS products concentrate in the operational level, the typical example of BRMS is assignment of rate to insurance policy, which is clearly an operational decision. What is the situation in event processing ? There is a famous analyst presentation that talked about "detecting threats and opportunities" as part of the ROI for event processing pattern matching. Let's examine what's behind this title. Sometimes there are risks in the operational level, such as security attacks, but since this presentation have concentrated in the business area and not the IT, the meaning has been seeking opportunities for the business and mitigating risks for the business, which is beyond the operational level, such detection is probably in the tactical level, but the outcome can flow to the strategic level, since there may not be an answer to specific threat or opportunity within the current strategy. On the other hand, event processing is also associated with being done on-line (what some people call "real time" or "near real time" when they are not sure if it is really "real time", which is even worse as a term).
Some interesting questions on this topic are:
1. Whether organizations are really doing tactical and strategic decisions on-line ? in the illustration on the top of the page taken from the Microsoft site
the authors believe that tactical decisions is matters of days to months, and strategic decisions are done in resolution of quarters or years. Is there benefits/feasibility to change it online ?
2. Do we need different variations of event processing for the different levels ?
3. Is the semantics of events the same among all levels ?
4. Are there different complementary technologies, and different platforms in the different levels, or can we look at a single event processing system across levels?
Today I am just posting the question, will try to address each of these questions in subsequents postings.
EPTS moved to an hyperactive phase with the launch of the six working groups. This has been also an opportunity to gain more members, as membership in the working groups is a privilige provided for EPTS members. There are several more members that belong to most of the different communities --- academic peo0le, consultants, customers and vednors (no new analysts...). I am especially glad to mention that CA has recently joined EPTS; I welcome CA as an expereinced company whose main line relates to managing infrastructure events, and knowing some of its people that have already been involved in the community, I am sure that they can make a substantial contribution to the technical society.
In fact, any new member interested to contribute to the understanding and progress of the event processing area on multiple fronts, is most welcome to join. There is a lot of action and place for contribution and innovation. Event Processing is a relatively young discipline; disciplines progress is accelerated with a community work. I am confident that the activities done today with a lot of excellent people who invest time and energy to contribute, will make a substantial impact on the universe. If you are interested to join EPTS, please follow the instructions
Somebody who looked at the public EPTS home page has asked me why there is not any trace for all the activities in this site? the answer is that on the public site we'll post final reports, but work in progress will be done internally, thus the "action" is reflected in the members Wiki.
EPTS is also starting to be recognized in the larger universe as a representative of the EP community. OMG mentioned EPTS explicitly in its RFP that deal with event meta-modeling stuff. Several articles in professional magazines, both industry oriented journals and general computing magazines are running articles about EP or CEP and cover EPTS as part of this article, I'll write more about any such article when I'll have it under my hands.
This is a slide rule, an ancient means to do arithmetic calculations easily if one has some experience in working with it. When I took the matriculation exam in mathematics many years ago, the Israeli ministry of education did not allow the use of calculator, since calculator at that time was relatively expensive, and it was considered of giving unfair advantage to those who can afford it, however they allowed the use in slide rules, so it served me well at that time. Today slide rules together with logarithmic tables and typewriter found their way to museums, but other type of rules are still with us.
Some Blogs have recently made references to the recent Forrester report with the catchy name:
Must You Choose Between Business Rules And Complex Event Processing Platforms?
The Forrester reports discusses some confusion that exists between the two terms. It is true that there is some ambiguity of the word "rules" - on one hand rule-based is a kind of programming style that can be used to express event processing patterns, and between BRMS - a collection of products with a certain functionality. Forrester also claims that to add to the confusion there are people who use (or abuse) BRMS products to do CEP applications and CEP products to do business rules applications. You can read the rest of the original report for more details. In my previous posting about state processing and event processing I have talked about the difference between the two. In fact, BRMS products are processing the current snapshot (state), while event processing is about processing of the history of transitions, different kind of techniques and optimizations are used for each.
I have also blogged recently about decision agents, talking about the fact that event processing agents (at least some of them) can be a subset of a larger whole which can be called - decision agents. And indeed, while the two type of technologies are distinct, there is also a sense to look at them with a unified view. Here I share the vision of James Taylor who talks a lot about Enterprise Decision management which consists of business rules, event processing and analytics. We'll hear much more about this concept - more later.
I'll start with a reminder that the deadline for submitting papers for DEBS 2009. The DEBS conference is the major conference for event processing related research. I would like to highlight especially the industrial track and solicit submission both from vendors and customers. Vendors may not submit a marketing oriented paper, however can send papers about architectures, technical questions, language issues, semantic issues, and engineering related topics. We especially solicit experience reports of event processing systems of any kind that are working in production, with experience insights that may be of general interest. Note that a short abstract is due next Monday, February 23rd, the deadline for submission is currently March 2nd, however extension of a few days will probably be possible.
Many years ago I have taken a course that has qualified me to be "IT project manager", and when talking about software life-cycles the instructor showed us the picture above, since then I have a Pavlovian Conditioning that the term life-cycle is associated with this picture.As a follow up to my recent posting about static and dynamic event flows, one of the related questions is whether event processing systems have the same life-cycle as other software artifacts, i.e. has to go through development and testing processes before going into production, or since it is very easy to modify such applications, this is not really needed, and changes can be done fast.
As usual in event processing there is no single answer. In some cases the event processing software is part of mission critical applications and thus any change should go through some validation and approval processes, which, alas, takes time and hurts the agility; in the other extreme there are systems in which an individual consumer defines personal alerts from events related to social networks, this application is not part of any enterprise application, and the consumer can add/modify/delete alerts every 10 minutes without upsetting anything except for himself, so there is no meaning to life-cycles and red-tapes.
Some applications may be hybrid. Part of it is a controlled logic, and part of it can be free for use to users.
