Is Business Intelligence dead?

Neil Raden posted recently an article entitled " BI is dead! long live BI".  This adds to the trend of declaring some technology or trend as dead:    Earlier this year I wrote a post in this blog, entitled: "Is mapReduce dying?" in which I also indicated some references to claims such as  "SOA is dead" or "ERP is dead". Actually nothing in the IT industry dies quite easily,   I guess that COBOL, which is about my age, is still alive  today, and may outlive me.   What Neil claims is that the success of BI was much less than the hype created around it, and its adoption was around 10-20% in large organization depends on the survey.   This is consistent with other opinions,  James Standen reported  on a survey that found that the the actual adoption is lot less than the percentage claimed  by BI vendors are. The picture at the top of this page is borrowed from his post "Business Intelligence adoption low and falling".     

Neil claims that BI does have a future, but its future is in doing it as part of a decision management framework, not as a stand-along service consumed by people.   When talking about decision management, it has a role to analyze historical data, but doing it within a context of a specific decision.  This can be used to provide input to decision or to reinforce or refute assumptions.  

I think that this is an interesting perspective and in a way has some similarity to event processing which also transitions from stand alone technology to part of a bigger game.   I have discussed it in the past, and still stand by the conclusion that stand alone event processing will stay as a niche, but the mainstream will be part of bigger games.  I have investigated recently the current list of "bigger games"and will write on it in one of the future posts,  however, I see the same observation as valid for BI,  current BI will remain a niche, while the mainstream will move towards embedded BI.   More -later. 

Book review: The Decision Model

The last package from Amazon brought me the book entitled: "The Decision Model: A Business Logic Framework Linking Business and Technology" by Barbara von Halle and Larry Goldberg.

I have read a draft of the book before, at Barb's request, and wrote a review, from which one line was quoted on the back cover; I believe that the trend of modeling decisions and look at them in perspective of higher level abstractions will become more pervasive, and I view technologies like business rules, event processing and various analytics as building blocks in decision platforms that are going to be notable part of enterprise computing and managing much of the operational decisions. The book has three sections:

Section I puts the decision model in context, explains what is decision model, providing a background comparing decision models with data models, and positioning decision model in the SOA and BPM universe, it also explains the business value. This section is intended mostly for business users and managers that want to get an overview.

Section II explains the decision model in detail, discussing the structural, declarative and integrity principles, and comparing the decision model to the relational model, a motive repeating in previous books by Von Halle. There is even a chapter that is called "The decision model formally defined", but the formalization is in terms of explanations and tables, and not by formal writing, which I guess fits the target audience.

Section III is called "Commentaries" and is actually a collection of articles by the authors as well as by various people active in this space (John Zachman, James Taylor, Bruce Silver and more) discussing specific related issues such as: relations to enterprise decision management, standards, business decision maturity model.

Event based decisions and event processing are mentioned several times within the book, but are not thoroughly discussed. The focus is on facts and rules kind of terminology; a combined model that combines both rules and events is a natural extension, from the point of view of this decision model as well as from the point of view of event processing modeling. I have written before about decision agents, and since that time advanced on the thinking about such a decision agents framework. I'll revisit this issue in one of the following postings.

Bottom line -- the decision model book is a very good book to explain the book to various types of readers (the introduction maps the chapters of the book to the various types of users) and possible basis for both pragmatic foundations of rules technology, as well as a possible basis for a more formal basis for extended decision agents framework. More on this topic -- later.

Getting closer to the peak of inflated excpectations

The Gartner hype cycle has a notion called "peak of inflated expectations", which states that different technologies go up in the hype ladder until getting to the peak, where people think that can solve much of the universe's problems, and then, somehow people realize that this is not the case and go through frustration and disillusionment, until realizing the true value (if any !), and getting back on track, now with the right set of expectations. Two recent Blog postings show some indication that we are getting closer when talking about event processing:

• Carol-Ann Matignon from Fair Issac complains in her Blog that some people say that event processing = decision management and brings some counter examples where there are decision management applications that use batch processing.
• Paul Vincent from TIBCO wonders whether CEP = continuous analytics.
  

Actually, both are right. Event processing may have a role in decision management, there are some applications of decision management that are pure event processing, some in which event processing has some role, but not doing the entire trick, and some that are really batch oriented data management. Likewise continuous analytics can be done in response to event (raw or derived) or just periodically. Event processing may or may not have a role in deciding when to do this analytics (e.g. optimize the traffic light setting), the optimization itself is typically not event processing per se.

I think that it is very good to observe that event processing can play a role in many areas, likewise, it is also good to be clear about its possible role, and which are the cases in which it has value, and which are the cases it hasn't, I guess that we'll have to wait for the enlightenment phase, in the hype cycle terminology until there will be more universal clarity about the role and value of event processing. More - Later.

More on Decison Management

Today is the last day of the 8 days holiday, and tomorrow - back to the office (well for one day only, since our weekend in Israel is Friday and Saturday, and Sunday is a regular working day), and I have a huge "to do" list that keeps accumulating, and a bunch of phone calls... Today, however, I have travelled with my family to Bet She'an, one of the biggest archaeological sites in Israel (around 70 KM from Haifa). The roman city was exposed and much of it has survived, as you can see in the picture above (there is much more).

Today - just a short posting as a follow up on James Taylor's posting, who is advocating for a while the notion of decision management. The following slide is copied from JT's posting and explains the difference between decision management and decision support




The difference, as shown, is between helping people to make manual decisions, which is the BI land, and make automated decisions, which is the DM land. One of the topics I am working on recently is the notion of "decision" as an abstraction which should be programmed directly by business users, and will hide a blend of technologies such as: BRMS, event processing and various kind of analytics; I believe that the next wave of computing will be centered around the ability to business users (or direct consumers) to program and the role of IT will be replaced by autonomic computing techniques -- we are not there, but this is something to expire. Event processing will play multiple roles in this setting, but I'll leave this discussion for another time -- it is late, and I want to advance in the book I am reading since tomorrow will be busy..

On Cool Event Processing

Thanks to a recent posting of Tim Bass, I have watched now a really cool video from the MIT Media Lab,
if you have not already done it, watch and enjoy ! still in early phases, but very impressive !

This brings us to two interesting questions:

1. Does this demo show an event processing application ?
2. Should creating cool applications be our target ?

As for the first question -- the main achievement of the MIT Media Lab video demonstration is the ability to point with the finger on some item (a person, a product in the supermarket etc..) and use image processing technologies to identify it, bring information from the Web, and screen it on the item itself (e.g. screen the Amazon book reviews on the book, screen annotations about the person on a person's body etc..). This is an extremely impressive blend of technologies, but not really an event processing. To me it looks as a request-response type of application and not event-driven. The action of pointing out an object is a request to identify it, which in turn sends another request to search the web. Not really event processing, but certainly very cool...

Which leads to the interesting question number 2 --- for sure, it is easier to impress and sell technologies through cool applications. Event processing has some cool applications in processing events in games, processing event in the smart house that automatically turns on and off the lights, re-stocks the refrigerator, and invites technician to fix the air condition. I think that the issue of event processing for the individual consumer market has not been investigated well, and in that context the "cool" stuff is certainly a good way to sell...
While looking at the majority of the work done today in event processing, it relates to enterprise computing, in enterprise computing the main criterion is ROI, there may be nothing exciting about an accounting, procurement or regulation enforcement applications, but since they are part of the enterprise's bread and butter, technologies that enable to the enterprise to do them more effective/more efficient may bring a lot of ROI. Since the decision makers are people, and decision making is not necessarily a rational process, cool demos are highly recommended...

More - Later.

On levels of decision makers and event processing - part I

I am sitting now in my living room, watching the heavy rain outside.Rainy; We did not get a lot of rain this winter, however when the rain comes it tends to come in the wrong times, like weekends (Friday is part of our weekend which is Friday and Saturday, Sunday is a normal work day, it is used to catch up on things since in this day our colleagues from abroad are idle and there are no conference call and other interactions).

Today I would like to concentrate on the question --- what level in the Enterprise is event processing for, I had a recent discussion with somebody who investigated the BRMS market and asked him this question about BRMS, and the answer was the mostly BRMS products concentrate in the operational level, the typical example of BRMS is assignment of rate to insurance policy, which is clearly an operational decision. What is the situation in event processing ? There is a famous analyst presentation that talked about "detecting threats and opportunities" as part of the ROI for event processing pattern matching. Let's examine what's behind this title. Sometimes there are risks in the operational level, such as security attacks, but since this presentation have concentrated in the business area and not the IT, the meaning has been seeking opportunities for the business and mitigating risks for the business, which is beyond the operational level, such detection is probably in the tactical level, but the outcome can flow to the strategic level, since there may not be an answer to specific threat or opportunity within the current strategy. On the other hand, event processing is also associated with being done on-line (what some people call "real time" or "near real time" when they are not sure if it is really "real time", which is even worse as a term).

Some interesting questions on this topic are:

1. Whether organizations are really doing tactical and strategic decisions on-line ? in the illustration on the top of the page taken from the Microsoft site
the authors believe that tactical decisions is matters of days to months, and strategic decisions are done in resolution of quarters or years. Is there benefits/feasibility to change it online ?

2. Do we need different variations of event processing for the different levels ?

3. Is the semantics of events the same among all levels ?

4. Are there different complementary technologies, and different platforms in the different levels, or can we look at a single event processing system across levels?

Today I am just posting the question, will try to address each of these questions in subsequents postings.

On BRMS and EP

This is a slide rule, an ancient means to do arithmetic calculations easily if one has some experience in working with it. When I took the matriculation exam in mathematics many years ago, the Israeli ministry of education did not allow the use of calculator, since calculator at that time was relatively expensive, and it was considered of giving unfair advantage to those who can afford it, however they allowed the use in slide rules, so it served me well at that time. Today slide rules together with logarithmic tables and typewriter found their way to museums, but other type of rules are still with us.

Some Blogs have recently made references to the recent Forrester report with the catchy name:
Must You Choose Between Business Rules And Complex Event Processing Platforms?

The Forrester reports discusses some confusion that exists between the two terms. It is true that there is some ambiguity of the word "rules" - on one hand rule-based is a kind of programming style that can be used to express event processing patterns, and between BRMS - a collection of products with a certain functionality. Forrester also claims that to add to the confusion there are people who use (or abuse) BRMS products to do CEP applications and CEP products to do business rules applications. You can read the rest of the original report for more details. In my previous posting about state processing and event processing I have talked about the difference between the two. In fact, BRMS products are processing the current snapshot (state), while event processing is about processing of the history of transitions, different kind of techniques and optimizations are used for each.
I have also blogged recently about decision agents, talking about the fact that event processing agents (at least some of them) can be a subset of a larger whole which can be called - decision agents. And indeed, while the two type of technologies are distinct, there is also a sense to look at them with a unified view. Here I share the vision of James Taylor who talks a lot about Enterprise Decision management which consists of business rules, event processing and analytics. We'll hear much more about this concept - more later.