On the right technology for decisions

I came across a (not new) discussion by my IBM colleague Jean Francois Puget  published on the IBM developerworks entitled  "What is the difference between SPSS and ILOG".  Actually, while Jean Francois colors it in blue and discusses it within specific IBM products, he says that he is more interested to discuss the generic question about what is the right decision technology, as there are various technologies today that are labelled as decision technologies, decision management and other kind of decision oriented names. 

Jean Francois makes the distinction between  "single decision at a time" and "doing a group of decisions together" and asserts that for "single decision at a time" BRMS and/or predictive analytics is the right kind of technology, while for "doing a group of decisions together" optimization techniques are appropriate.  

This has some truth to it,  but I am not sure that it is the ultimate differentiation between these two types, so let's look at this issue.   When there is a need to do a decision, there are several approaches:

1. Get a person all relevant data and let this person do the decision
2. Make automated decision (or recommendation)- when the way to do the decision can be  codified as decision trees/decision tables/rules
3. Make automated decision (or recommendation) --  when the decision needs to find the best alternative according to a quantified criteria. 

For each of these cases, the data obtained can be deterministic or stochastic, existing or predicted, and there are various ways to achieve this type of data, but this is true regardless of the three cases.    It seems that approach 1 does not require any decision technology - although some people call requested data that uses some kind of inference technique also a decision technology, but I think that it might be taking the term decision to non-intuitive place;  approach 2 requires some kind of rule technology, and approach 3 requires some kind of optimization technology. 

Now,  there are cases in which single decision requires optimization. For example, a person wins the lottery and needs to get a decision where to invest the money. This is a single decision with a lot of alternatives, it requires also predictions on these alternatives,  and the person has some objective function and constraints on types of investments.   There are cases in which there are multiple decisions that have to be done at the same time -- for example:  who should receive bonus, however the amount of bonus recipients is fixed, and the criteria are very simple, so no optimization should be done, just a lot of rules applied to all candidates to rank each of them, and then sort by the ranking.  So there is selection between alternatives, but optimization is not really required.   While there is some correlation between the criteria specified by Jean Francois and what I have written here, but it seems to me that the main distinction is what is the kind of decision, and the way alternatives are compared...   More on this - later.

Decisions in smarter systems

Arlington., Virginia,  Hayatt Hotel

I am here for the OMG technical meeting.      I have participated in (part of) the decision modeling day organized by Paul Vincent and Christian De Sainte Marie.  Their ultimate goal is to get to a standard on decision modeling, and they have issued proposal for RFP on that issue.  A good survey of that day can be found in James Taylor's Blog.  I sat near James, and he is blogging in real-time.    James himself gave an interesting keynote on the 
importance of decisions  


James concentrated on operational decisions and said in many of the organizations the role of computerized systems is to provide data (in various ways) to manual decision makers when they ask for it.    The smarter systems have larger portion of automated decisions, they are active rather than passive - determine when a decision is needed, and the decisions are measurable with quantitative metrics, so they can be evaluated.  


While James did not talk explicitly about event processing,  it is obvious that it has a significant role in his vision, it has several roles:

1.  determine when a decision is needed
2.  the automated decision itself is often context dependent and  the context can be determined by event processing context mechanism (temporal, spatial, event history related...)
3. the decision itself may depend on event pattern
4. Last but not least -- the extension of event processing to proactive computing coupling with the metric that measures the decision's result can trigger decision to mitigate undesired predicted deviation from the result,(I discussed this one with James during the reception in the evening).  

The EPTS virtual symposium - tomorrow.   A lot of logistics to get it running! 

On events and decisions

Recently there are a lot of traction around events and decisions, coupling events and business rules, talking about decision platforms and decision servers,  meta-modeling efforts and more.  Some blogs dealt with the relationships between business rules and event processing - such as: James Taylor, and recently Paul Vincent. 


It is interesting to try and classify all the relationships between events and rules,  this is a rather streightforward list, no big insights:

1.  An event trigerrs a notification to a person for manual decision.
2. A derived event that is generated by an event processing system using filtering, transformation, aggregation, pattern matching or any combination of the above;  the derived event triggers a notification to a person for manual decision.
3. The next phase in the evolution is that raw or derived event triggers an automated decision based on either predetermined rules  or triggering an optimization process that generates a decision.\
4. In some cases the event processing system is the one doing the decision, since the pattern matched is the essence of the decision (e.g. who won the bid? ) and the event processing system itself triggers an action which is the result of the decision.
5. The opposite direction is that decisions also emit events:  events can be emitted to track the decision process in case it is multi-state decisions (e.g. procurement approval),  to check consistency among decisions, or consequences of this decisions in light of other events, and to match decisions with their consequence events in order to evaluate the quality of the specific decisions.
6. Events can also trigger modifications in the decision process, e.g. tuning parameters in policies, change the rules' segmentations and more

There may be some more interactions,  but these seem to be the main ones. 

On Decision Agents

Decisions are part of the enterprise life as well as the life of every individual. Take a decision that many of the readers have experienced: naming children. My wife and myself have realized in very early phase that we have completely different taste in names, so we have decided to agree on a protocol for how names are selected: taking turns - one of us is making a list of five names, and the other selects a name from this list. The selection is done only after the birth, and the list can be modified until the selection made. To be fair we need even number of children so each will play any of the roles equal amount of times (indeed we have four children which satisfies this requirement). This is actually work, none of us got the first priority, and none of us have to tolerate a name we really hate.

In the paragraph above I have described a manual decision, but increasingly decisions are made by computer software which makes or recommends decisions. James Taylor is constantly talking about "Decision Agents". I thought that it will be interested to look at this notion and discuss how it is related to event processing.

We can say that a "decision agent" has various properties that we can qualify as answer to questions:

• Why ? what is the reason that the decision agent activated. In the example above -- the birth of a child.
• which ? which information is needed in order to make this decision. In the example above --- the list of five names (which have been obtained by another decision or collection of decisions) .
• How? How the decision is made. In the example above --- by some human cognitive process, which may have reflection in a computerized decision agent.

Back to our world of event processing, we are using the term "event processing agent" to denote a software artifact that gets one or more event as an input, does something, and produces one or more events as an output.

The question is what is the relationship between decision agent and event processing agent ?


In a (not very recent) postings, Carole-An Matignon from Fair Isaac has attempted to demystify some terms. She used an analogy to the human body, saying that BRMS is the brain, while event processing is the sensor for the brain to get the decisions. So is event processing agent is a sensing agent ?

The answer is --- the terms decision agent and event processing agent do intersect, but none of them subsumes the other.

Returning to the decision agent questions.

The why question: A decision may be required since some situation has occurred, because some relevant fact has changed, or because somebody made an explicit request to activate the decision agent. In the first case (a situation has occurred), then this situation may be a simple event, but also may be a leaf of an event processing network. In this case, there may be some event processing agents that are part of the decision to activate the agent, so it is part of the brain.

The which question: Here again the information needed can vary -- it may relate to the present state, to the history of states, and transitions. Event processing agents can be used to prepare the required information, by taking events and filter, transform, enrich, aggregate, split and more. In this case the event processing agent is indeed a sensor, creating input for the decision.

The how question: There are various techniques to get a decision, detecting patterns on the event history may be a method to obtain a decision, together with other techniques, such as inferring from facts and rules, applying stochastic decision reasoning and more. This does not say that every event processing agent which performs "pattern detection" is indeed a decision agent, sometimes it just derive event that will be used directly or indirectly as input to a decision.
Interestingly, this is true for business rules as well. A business rule may derive new fact, the fact itself is not a decision, for example, it may be classification of a customer based on demographic information. The output of this rule is used as input to (another) decision agent. So BRMS like event processing can also play the role of both sensors and brain.

To summarize:

• An Event Processing Agent may be a Decision Agent, or provider of input or trigger to other decision agents.
• The same statement is also true for "state processing" business rule.
  
• A decision agent may be Event Processing Agent, but also can consist of several other types of agents.
• There may be blend of decision agents of various types inter-related. I'll write more in the future about this assertion.
  

On external and internal decision in event processing

This picture shows the external and internal parts of an house (not mine...). I am at home now in a day off (have to finish 10 days by the end of the year so taking them one by one), have to travel to Tel-Aviv on some matter, but meanwhile spending a few minutes in Blogland. My last posting has dealt with event processing and decisions.
James Taylor has answered in saying that decisions may be triggered by events, but the decision itself is not dependent on the events. I think that we are on the same wavelength here, so let me make some comments in my terminology. In the early days of active databases there has been a distinction between its use for "internal applications" that helps to manage the DBMS system itself, i.e. manage transactions, and "external applications" which belong to the business domain. Likewise, we can observe internal and external decisions in processing events. The event processing network consists of a lot of internal decisions -- decision of where to route events, decision of which events will be filtered out, decision on when context should expire, decision on whether a pattern has been detected, a decision what to do when a pattern is detected and more. This are all internal decisions that help moving the EPN towards its edges. The interface between the EPN and its consumers occur at that edges. In the edge there are several possibilities:

1. The EPN notifies the consumer about the event that flows in the edge, and the consumer decided manually what should be done, or can manually run any decision process

2. The EPN triggers a decision rule, the decision is a function of the event that triggered it, in this case the EPN triggers an external decision.

3. The EPN triggers a decision, but the decision is not a function of the content of the triggering event, this I assume is the case that James Taylor has referred to.

On Decisions and Event Processing

Saturday morning at home, after busy week and one resting day (our weekend is Friday and Saturday, tomorrow - Sunday, is again long and busy working day...) I am back in the Blogland.
Today I am in reactive mood, so will join a discussion thread that started by James Taylor who seems to have discovered that "event rules" and "decision rules" are not the same. Since this has been a reaction to a previous posting by Marco Siero, Marco actually agrees with him, saying that event rules and decision rules are not the same - saying that the "event rules" are in fact activation rules - saying WHEN a decision is needed. If they both agree then the thread of discussion can end.. -- nevertheless, I have some comments to this discussion:

1. The word "rules" is overloaded - there are many types of rules. Some rules deal with obtaining more facts (if the rules are based on logic it is called inference, and if the rules are based on data models it is called derivation), some with activating actions. Moreover, there are rules which are assertions about what is allowed and what is not (constraints) which are none of the above.

2. People have used the term "rules" to denote -- detect a pattern in events --- due to several reasons, one of them seems that rule has been the most closely known term that they could anchor at.

3. Today, when "event processing" is known by its own right, the use of term rules, when it is not needed is just confusing, and it is better to use the correct terms for event processing types.

4. I personally used the term "rules" for several years, but in the last three years have avoided doing it. It is difficult to get people change terminology, but sometimes it is required.

This, however, was just a comment on the term "rule", but let's get to the more interesting part -- as the title of the posting hints -- relationships between decisions and event processing.

The active database people have coined the term "active rule" which is also known as ECA rule, following its parts (event-condition-action). One can look at event processing network as an extension of ECA, in the sense that there is an entire processing done to get to the event that we actually want to react to, a raw event can be filtered, translated, aggregated, enriched, and
combined with other events that satisfy a certain pattern -- at then the event at the edge of the network (AKA situation) is the one that triggers the action.

The question whether an event processing application is doing decision or just notification that decision is needed, is not dependent on the event processing part - but on the type of action, if the action is notification -- notify somebody that the situation has been detected, I guess that it is what Marco meant in "activation rules", however, if the action is an automatic action (or several possible automatic actions based on conditions), then the result of the event processing is certainly a decision. There are cases in which the event processing ends in notification, e.g. most cases of BAM, and there are cases in which event processing ends in automated actions, e.g. send order to renew inventory, block a certain financial transaction and many other examples.

It also should be noted that the process of "event processing" may include decisions about the detection process itself -- examples: diagnosis, fraud detection - require application of decision components in order to determine what is the situation that occurred, which is also manifests itself in the decision.

James Taylor makes more interesting assertion: Event rules are not the same as the rules behind business decisions, even if the same language can be used to express both. One is tied to the events and the event handling system, one is not. I am not sure what is "tied" to the events, as said, the "action" takes part outside the event processing network, and it certainly can be a case of hybrid system, in which an event processing network detects what happened and the action attached to it is a "business rule" that decides what the reaction should be based on the situation. More- Later.