On enrichment - and the difference between BRMS and EP

A recent article in the IBM developerWorks discusses two ways to enrich data used for rules from external databases,  one of them is doing the enrichment in the request level, before calling the "decision server" (which is the current name for using BRMS system using the request-response protocol), the other one is doing enrichment during the rule processing itself. The article describes how each of these options is done and also discusses pros and cons,  the benefits of enrichment by the request level are - less complexity, and better performance of the rule component; the benefits off enrichment at the rule level are - handling dynamic data and more specialization for the exact data that is being used by the rule.


Thinking about event processing -- there is similarity to the BRMS case,  event can be enriched both by the event producer and as part of the event processing itself, the arguments are not far from those in the BRMS case,  there is one fundamental difference in event processing --  the work is not done using the "request-response" protocol,  moreover, the different part of the system are decoupled, thus the event producer does not necessarily know what purposes the event is going to be used, thus there may be different types of enrichment needed for different uses.   The dynamic aspect is applicable here, and there may be some race conditions in highly dynamic systems between updates in the database that was enriched and its use in enrichment, unless the event processing enrichment system locks the data in the database until it is being used in the event processing system, which requires the event processing system to exhibit a transactional behavior for part of it, but I'll not get now into this issue,


Bottom line:   The considerations in event driven architecture are somewhat different than the request-response systems that are the most common one in computing.   

On reasoning about events and states

Yesterday, we watched in our local theater, the play "Amadeus". I have seen the movie when it was first launched, and another version of this play in the nineties. Still very impressive, and an opportunity to hear some of Mozart's good music.

Thanks to Paul Vincent's Blog, I clicked the link and got to Paul Haley's posting entitled "time of the next generation of knowledge automation". Paul Haley starts his postings by classifying four types of reasoning:

1. Reasoning at a point within a [business] process
2. Reasoning about events that occur over time.
3. Reasoning about a [business] process (as in deciding what comes next)
4. Reasoning about and across different states (as in planning)

The claim is that the first kind is being supported by business rules (called EDM in the original), the second kind is being supported by event processing, while the third and fourth kinds are not really supported by the state of the art. The third type requires getting the notion of context into reasoning while the fourth type requires getting cross state reasoning (which is different from cross event reasoning). I agree with the classification, and will write more in the future about the third and fourth types in depth. Actually, we also need reasoning that will combine the different types of reasoning as well.

On Declarative programming in event processing

The city of New York has issued a 3 cents value stamp for its 300 anniversary, I am not in the stamp printing business, and wonder how long stamps will survive, but the blogger tool tells me that this is my 300 posting on this blog, in almost two years. Since recently I am dealing in languages, today I'll share some thoughts about declarative programming and its implications on event processing. John Bates told in his DEBS keynote about his experience, in the assumptions they had when developing Apama in the lab, and how these assumptions were slightly changed when got out of the lab. This is a phenomenon that I know well from a few times over my life that I have been involved with getting ideas from the lab to reality. Of course, the closer that your assumptions made in the lab are to reality, the faster time-to-market you have, and better success probability. Among the opinions that I used to hold is that declarative programming is always superior on imperative programming, in imperative programming, people need to work difficult needlessly, while in declarative programming, programming is more elegant and easier. I did not really changed my mind, however, reality shows that its ain't that clear cut. Getting to a neighboring area, it is well known half-secret in the business rules industry, that while all BRMS products support RETE style declarative inference rules, most users prefer to use a more imperative looking "sequential rules; I think that the reason is that in inference rules the flow is hidden, each rule is defined individually, and the flow is derived from the fact that some rule derives some fact, and some other rule has such a fact in its conditions, but this makes all the programming as "programming in the small", and the "programming in the large", controlling on how various rules relate to each other, does not exist in this programming style. It turns out that it is important for users to have visibility, and even control on the programming in the large aspects. Coming back to event processing, I believe in a combination of declarative programming for the programming in the small, i.e. to define any individual event processing agent, and making the programming in the large, i.e. the event processing network, explicit. This is not really an imperative model, since the EPN can be implemented in various ways, but it is not pure declarative either. We see that this type of programming model is getting traction.
I'll discuss the various dimensions of event processing languages in depth, in further postings - stay tuned.

Here is a picture I got recently from Matty Cooper, from EventZero, who made all the way from Australia to Nashville to attend DEBS 2009. This picture was taken during the event processing language tutorial

On BRMS and EP

This is a slide rule, an ancient means to do arithmetic calculations easily if one has some experience in working with it. When I took the matriculation exam in mathematics many years ago, the Israeli ministry of education did not allow the use of calculator, since calculator at that time was relatively expensive, and it was considered of giving unfair advantage to those who can afford it, however they allowed the use in slide rules, so it served me well at that time. Today slide rules together with logarithmic tables and typewriter found their way to museums, but other type of rules are still with us.

Some Blogs have recently made references to the recent Forrester report with the catchy name:
Must You Choose Between Business Rules And Complex Event Processing Platforms?

The Forrester reports discusses some confusion that exists between the two terms. It is true that there is some ambiguity of the word "rules" - on one hand rule-based is a kind of programming style that can be used to express event processing patterns, and between BRMS - a collection of products with a certain functionality. Forrester also claims that to add to the confusion there are people who use (or abuse) BRMS products to do CEP applications and CEP products to do business rules applications. You can read the rest of the original report for more details. In my previous posting about state processing and event processing I have talked about the difference between the two. In fact, BRMS products are processing the current snapshot (state), while event processing is about processing of the history of transitions, different kind of techniques and optimizations are used for each.
I have also blogged recently about decision agents, talking about the fact that event processing agents (at least some of them) can be a subset of a larger whole which can be called - decision agents. And indeed, while the two type of technologies are distinct, there is also a sense to look at them with a unified view. Here I share the vision of James Taylor who talks a lot about Enterprise Decision management which consists of business rules, event processing and analytics. We'll hear much more about this concept - more later.