Seperation of concerns and Event Processing Language

In one of the previous blogs I have cited the "hammer and nail" blog written by Louie Lovas from Progress and said that I don't have the honor to know the author personally, so he has sent me a note introducing himself - very nice of him to do that ! Mr. Lovas recently has written another blog entitled: "Hitting the nail on the head", in this blog, the opinion is that the current trend is separation between the presentation layer, logic layer and data layer, and that using SQL for event processing is going backwards - since it mixes the business logic layer and data layer - if I understood the reasoning correctly. I agree with the end result of the author about the fact that SQL may be inappropriate language to express some event processing patterns, and I have written about it before, I also big fan of "separation of concerns" and other separations as a software engineering and architectural practice. However, these two things are independent, in the past, when we worked on active databases, this has been indeed under the concept that business logic is embedded inside the database logic, but, this has nothing to do with the API. We can still have separation to architectural layers on one hand, and share the same API, on the other hand; there are even some benefits in doing that - such as reducing the number of APIs. Thus - from the architecture point of view, I don't see any problem in using SQL-oriented language for event processing; the same (or similar) API can appear in various architecture layer (and in various roles).


the problem, as said is pragmatic, and stem from the SQL semantics and its fitness to event processing. In fact, in the course I am teaching we'll probably do with the students some experimentation on that.

More - Later

Teaching an event processing course

Since nobody ever took a picture of me teaching a class - I have brought the closest thing, a picture of me giving a talk in a conference. Anyway, this week I have started to teach an event processing course at the Technion. It is a strange situation now, since the faculty members are striking, they did not open the school year, but since I am an adjunct professor (the official title is: adjunct senior teaching associate) I am not striking, but they may close the campus at some point - so not sure what will happen. In any event, I have started the event processing course, and asked the students to read the blogs in this area as a background material. The course is intended to be a blend of theory and practice, where on one hand I am teaching the theory of event processing (my interpretation of it, to be exact), and one the other hand the students will be required (in teams) to implement a project that uses some CEP engine and some infrastructure around it to demonstrate an application prototype. Going to be interesting, and I'll provide more details about things related to the class as the course progresses. The discussions will also provide me more topics to blog around. One of the EPTS pending activities is to create a portal for teaching material around event processing. I know that some of my colleagues around the globe like Mani Chandy and Rainer von Ammon are also teaching event processing courses, and there may be perhaps some others. So will try to get some updates on the other courses as well. More - later.

Agnon, the dog, playing and downplaying

My literature teacher in the 7th grade has told the class one day a story about her days in as a university student; one of the literature classes in the university has been dedicated to Agnon, the only Hebrew writer who received Nobel Prize in Literature (Agnon's picture is in the left hand side). One of the famous stories of Agnon has a dog (such as the one in the right-hand side) as its hero; thus, the class of students at the university had discussed this story and had a debate about what does the dog represents, one group of students thought that dog is a symbol for Agnon itself, and one thought that the dog is a symbol for the entire Jewish people. Since Agnon was still alive those days, they have decided to send a couple of students to meet Agnon, set up a meeting in his house (well - Email questions were not on the radar at that time), and explained the dilemma - what does the dog represent? Agnon was somewhat amused to hear about the debate, and told them briefly - "I don't know what you are talking about, the dog represents a dog".

Going back for the childhood Reminiscences to the contemporary issues - I have read with interest the recent blog of Paul Vincent about "CEP vs. Business Rules" in which he mentioned me by reference (and not by name) saying: "Others seem to downplay the idea of rules for CEP". I have a substantial respect for Paul, and generally agree with most of the assertions he made in that blog. Let me add some more insights into this issue - I am using this blog to reflect about event processing, and make the point the "event processing" is a discipline that stands on its own feet, and is not a footnote to anything else - be it databases or business rules.

CEP functionality can indeed be implemented in either SQL or in an extension to business rule language as some of the vendors successfully demonstrated. It can also be implemented in half a dozen different ways (at least). Since I am trying to investigate "event processing" as a discipline, and not a specific implementation, thus is the play, and indeed I downplay specific implementations for now, and try to think on the fundamental issues. I may return in future blogs to talk about what type of implementations fit different types of functions/applications - which is a difficult question, and I don't really see that there is an issue of "CEP vs. Business rules" as these are conceptually orthogonal issues (and I agree that in practice there are various relations), and Paul's blog does a tremendous job of highlighting the differences and relations between the two.

For now - my dog is "event processing" and it represents "event processing" and not other concepts that are not fundamentally necessary to understand what "event processing" is.

More - Later.

Fallacies in implementation of event processing solutions

Back in Israel... the flight that has taken me back home suddenly landed in Athens, instead of continuing to its destination, it turns out that one of the passengers left a laptop on a vacant seat and went to sleep elsewhere in the aircraft, some passengers attracted the attention of the crew, and after some consultations, they decided to land in the nearest airport to have security people check the laptop, after landing the laptop owner woke up and retrieved his laptop, and the aircraft could take off again (well - not so fast, took almost an hour to take off again) - strange story (has been in several variations in all daily newspapers in Israel)... a fallacy in (human) event processing system. Talking about fallacies, recently Hans Glide in his blog talks about "failure scenarios" where EP solutions may fail to produce correct results. This reminds me of previous work I have done in the area of derived data using active databases, where in a relatively old paper I have looked at several possible fallacies in active databases - the analysis is also valid for event processing, here are some examples:

1. non deterministic execution due to conflicting rules.

2. Inability to reconstruct the derived data from the raw data since multiple rules derive the same data-element

3. Redundant updates in case of inter-connected derived updates, e.g. the same data-element is updated several times within one path

4. Derivations may result in infinite loops

The paper also shows that it is possible to eliminate these problems using a specific execution model.

Taking it back from active databases to event processing - in active databases, this paper has been in the framework of deriving data, while in the event processing cases we derive events - however, some variations of the same phenomena can occur in event processing - certainly non deterministic behavior can happen, and some other fallacies as well.

Hand Glide mentions that each event processing implementation project should pay attention to these - and indeed event processing application debug is more complex than the debug of regular applications due to inter-relations between rules/queries/derivations.

IMHO - it is a responsibility of the EP vendors to provide semantic debuggers that will enable doing it. Some are already around, but more work is done in this area, the users should not be on their own here... more about validation of EP applications - in one of the next posts (I am promising many follow-ups, but new topics keep coming)...