More on spatial event processing

This is a map of my home city, Haifa, which shows it northern part, I live in the south part outside this picture. I had some internal discussions this week about the role of location information in event processing, this is somewhat similar to the roles that time has in event processing, though the use of time is more pervasive, we see increasing number of applications that require spatial support. Some of the major utilizations of location information are:

1. Location as a context: The location is used to group events together as they serve as input for any event processing function such as: aggregation of pattern matching.
2. Location as enabler for spatial patterns: The locations of events is used in the function that determined whether a certain pattern is being matched.

Note that these two roles of locations are orthogonal.

An example for the first role: count all the police vehicles that are within 5 KM from a crime scene, where there are periodic events that identify the location of the police vehicles using GPS devise. In this case, location is done to find out which events are "in context", and the event processing function itself is a simple aggregation (count).

An example for the second role: A certain company experienced over one week, 20 events of breaking into employees houses, it an an attempt to figure out whether this is a work of a single person, the pattern that is matched is: the maximal distance between two breaking in events that occurred within a single night is less than 15KM. In this case the context is not spatial, since the events involved are determined by time ("a single night") and segmentation ("events related to houses of employees"), but the pattern that we are looking for is spatial, since it involves location.

As said, time can also play in the context role, in the pattern role, in both, or in none.

More on spatial event processing - later.

Towards an event processing manifesto - take one

The word manifesto brings to many people an association with the Communist Manifesto by Marx and Engels, which is an example how manifesto can be used, reused and abused. The term manifesto has been adopted by the computing community, and we have seen variety of works entitled: manifesto, some of them are: object oriented database manifesto, the manifesto for agile software development, and the business rules manifesto - just to name a few examples.

In May 2010 I am co-chairing (together with Mani Chandy and Rainer von Ammon) a Dagstuhl seminar, whose task is to compose "event processing manifesto", which we strive to make it as influential effort. A Dagstuhl seminar is a five days retreat in an isolated place in which a selected group of people are convened to make deep discussions about a certain area in computer science, many of these seminars are geared towards the academic community, but it this particular one we strive to keep the balance and have half of the participants from industry (vendors, independent consultants, analysts, and some innovative users), the number of attendees in each seminar in limited, and there is a big competition on this resource (and long waiting list after approved, we have put the application to do it in 2008). Here is a picture of Schloss Dagstuhl (located in Saarland, Germany):

I guess that I'll write a lot about the Dagstuhl seminar, the event processing manifesto, and related stuff in the next few months, so this is the modest starting point.

The intention of the manifesto is simple (but not easy) -- define what event processing is, what is the scope of event processing, are there mandatory features in event processing, what are the various options and variations, what are the synergies with other disciplines, and have as an appendix a call for action for the community: what are the research challenges and what are the pragmatic challenges.

I'll start today with some of my own thoughts on the scope issue.

I view "event processing" as a name of a computing discipline like "data management", "image processing" and "information retrieval" - relatively wide disciplines that cover wide collection of applications. This is also true for event processing. Alas, we still have some misconceptions even on the scope, I have used before the metaphor of blind people touching various parts of an elephant and concluding that each of them has identified what an elephant is, so this time I'll use another animal - ant whose sight is limited to its immediate environment.

I am using ant, since one of the great Hebrew poets has written a poem saying (in free translation): "my world is narrow as the world of an ant". Some examples of it:

1. people who view event processing as aggregation of time-series events, this is definitely an event processing application, but not the only one;
2. other people are looking at detection of anomalies within event collections, where the anomalies types are not predefined as what event processing is all about, this is indeed another event processing application, but not the only one;
3. yet other people are looking at real-time matching of predefined patterns with streaming events, which is somewhat different from the two previous examples.

One might claim that there is no event processing discipline at all, but there are very distinct areas that process events in different ways for different purposes, probably using different technique; while this may be a possible conclusion from the Dagstuhl meeting, my experience shows that there are many applications that require blend of event processing functionalities, and that the space is not discrete, but has some continuity, thus event processing in the wide sense should deal with all of them (and some more) and the synergies between them. So this is a point for discussion.

While the amount of people that will be involved in the actual composition of the manifesto is restricted, we'll strive to disseminate the draft to a wide discussion, both within the EPTS community and other avenues. As a comment, the Dagstuhl seminar is not directly related to EPTS, many of the attendees are not EPTS members, and some are members of adjacent communities and not the event processing community, however I think that EPTS should have a role in follow-ups.

On proactive behavior and application performance management

David Mavashev from Nastel has pointed me to his posting entitled:"who owns application performance management", it deal with the issue of whether operations group or application group own the SLA type performance management, and gets to a conclusions that this is being handled reactively instead of proactively. This observation is true for other areas as well, in most services operations, when a problem is created, it is solved in a reactive way; if in the supermarkets the line in the checkouts become too long, they add more checkout personnel, the same goes in service centers. Move to more proactive behavior will save time, money and getting patient-less people like me from getting high blood pressure. The use of proactive systems is still in its beginning, event processing is a key part in it, but there are some missing pieces to make it proactive, as the current technology is geared towards reactive systems. More about proactive extensions to event processing -- later.

Book review: The Decision Model

The last package from Amazon brought me the book entitled: "The Decision Model: A Business Logic Framework Linking Business and Technology" by Barbara von Halle and Larry Goldberg.

I have read a draft of the book before, at Barb's request, and wrote a review, from which one line was quoted on the back cover; I believe that the trend of modeling decisions and look at them in perspective of higher level abstractions will become more pervasive, and I view technologies like business rules, event processing and various analytics as building blocks in decision platforms that are going to be notable part of enterprise computing and managing much of the operational decisions. The book has three sections:

Section I puts the decision model in context, explains what is decision model, providing a background comparing decision models with data models, and positioning decision model in the SOA and BPM universe, it also explains the business value. This section is intended mostly for business users and managers that want to get an overview.

Section II explains the decision model in detail, discussing the structural, declarative and integrity principles, and comparing the decision model to the relational model, a motive repeating in previous books by Von Halle. There is even a chapter that is called "The decision model formally defined", but the formalization is in terms of explanations and tables, and not by formal writing, which I guess fits the target audience.

Section III is called "Commentaries" and is actually a collection of articles by the authors as well as by various people active in this space (John Zachman, James Taylor, Bruce Silver and more) discussing specific related issues such as: relations to enterprise decision management, standards, business decision maturity model.

Event based decisions and event processing are mentioned several times within the book, but are not thoroughly discussed. The focus is on facts and rules kind of terminology; a combined model that combines both rules and events is a natural extension, from the point of view of this decision model as well as from the point of view of event processing modeling. I have written before about decision agents, and since that time advanced on the thinking about such a decision agents framework. I'll revisit this issue in one of the following postings.

Bottom line -- the decision model book is a very good book to explain the book to various types of readers (the introduction maps the chapters of the book to the various types of users) and possible basis for both pragmatic foundations of rules technology, as well as a possible basis for a more formal basis for extended decision agents framework. More on this topic -- later.