Revisiting the Blog ** 2 again

I have never been in Arkansas, and in return - nobody from Arkansas have ever read my Blog - not surprisingly. What is surprising is that in the rest of the USA states (with the exception of North Dakota) have people who live there, or at least visited there and have read this Blog.

I have done some investigation on - who reads this Blog, when the Blog has hit 1000 distinct readers. I have decided to re-visit the statistics if the Blog ever hits 3000 readers - when I checked it this morning I found out 3217 distinct readers, so it is a good time to look at some statistics, actually this number is quite surprising, since I don't really think that there are so many people interested in this, and indeed some are probably passing by in the Internet roads, however the statistics show that around 45% returned more than one time. Moreover 569 people has visited the Blog more than 50 times (172 of them more than 200 times, where the number of Blog postings did not reach 100 yet), so I guess that this is provide the size of the community who really reads the material.

From Geographical point of view, it seems that I still writing mostly for the USA readers - 1530 distinct readers (almost half of the readers). In number of accesses - USA is the first, UK second, Israel third followed by- Japan, Thailand, Canada, India, France, Spain and Germany. However, when looking at the number of "distinct readers" - it is somewhat different order -- USA is still first and UK still second, but here Canada is third followed by - India, Israel, Germany, France, Australia, Holland and Sweden. There is coverage of all continents - some of our neighbors - "the Palestinian Territory" that does not have map yet in Google analytics, so I don't know where exactly, Egypt, Lebanon, Lybia and Iran are all represented. from South America - almost all access are from Brazil, some sporadic ones from other countries. In Africa - South Africa and some countries in the north - but also - Sudan, Kenya and Uganda. Also coverage of almost all countries in Europe and Asia - total 92 countries. As far as cities - the three big cities, in number of accesses are: Petach-Tikwa (in Israel), London and Bangkok;

the big cities in number of distinct readers are: London, New York City and Bangalore.

The biggest source of reference is Google in various ways, followed by other Blogs like - the CEP Blog, TIBCO's Blog and Rulecore's Blog and from David Luckham's site

The most popular postings were:

1. Agnon, the dog, playing and downplaying in which I stated the opinion that Business Rules are a possible way to implement CEP, but not the only way - which lead to several more postings investigating the relationships.

2. On Event stream Processing in which I stated my controversial opinion that the term "event stream processing" does not represent anything interesting, and the person invented it, Mark Palmer, thinks it should go away.

3. On the mythical event per second in which I stated that "event per second" really means nothing outside the context of a specific benchmark. I have dedicated more postings to this issue.

4. On bitter pills , a recent posting that answered Tim Bass' criticism about the state of the EP market, provided more optimistic view (followed by some constructive assertions in the next posting).

It seems that the popular ones are the macro-level - but I'll continue from time to time also to write on micro-level things, since it seems to have its audience also. The most popular to those who got through search engines were related to XTP and context terms - so I need to get back to them also.

So - thanks to all the readers -- more event processing related postings - later.

More on Standards and Event Processing

I have written about standards before a few months ago, and nothing seems to have happened since that time. As I was asked to talk in the OMG meeting in March about the role of standards in EP, I'll have to devise a more detailed view - but I still think that standards is one of the key issues we should invest in. In my opinion the most important standard is the language standard, the EPL - which should standard as a meta-language. There is an intention to start working on it later this year. Other areas - interoperability, modeling, various standards for event structure (header, payload). The importance of standards stem from various perspectives.

1. The customer's perspective -- the main competitor of EP COTS is hard-coding the functionality within a regular programming, and making EP as a non-issue. One of the claims of enterprises is that their developers know how to program in standard languages, and they will not invest in teaching them proprietary languages. While not everybody take this approach, the industry does not like proprietary. There is an effort to make Stream SQL language - but since it does not cover the entire market (even not most of it) - this is not enough.
2. The ability to draw incremental knowledge -- as an example, work in the academia on query optimization around SQL has helped the entire industry. Concentrating around a single language can serve as a focus and have the knowledge be incremental instead of disiributed.
3. When moving to the "platform" oriented EP, instead of the "engine" oriented of the first generation, a platform will be able to include various implementations - agents that are based on various technologies/engines... thus, various technologies need to inter-operate, but also be part of the same application, and we don't want the application be built in a mix of languages...

more -later.

Hype vs. value -- a constructive view

Besides free advertising to an e-book, the illustration above - together with the title indicates that I am going to write something constructive - in the previous posting on "bitter pills" I have provoked the evil spirits and got sick with a flu, so with some almost sleepless nights, I have been too tired to Blog -- today is much better - so back in business. There are several people complaining about -- we need to get from hype to more impact on business, something is lacking today etc --- all true ! I also said that we need to take a constructive view - not only complain, but see what should be done. So here is an initial attempt to do it, actually nothing new here, just listing some observations done by various people:

1. More packaged applications based on EP technology should be constructed -- the model of using an EP tool as an application development tool covers only small part of the potential market, since, in the end of the day, business executives are interested in applications and not in technology.
2. Enable business users (non-IT developers) to control the behavior (e.g. define, modify, compose patterns/rules), since agility is an important expectation of the customers
3. Learn how to articulate the business vale -- yes we all know "threats and opportunities" but this is somewhat too abstract for the decision makers -- the business value is also not unique, there are various types of applications with different business values, and explaining the right one is crucial - more thoughts about the various types of business values -- in one of the next postings.
4. Advance on standards -- customers don't like proprietary.

Of course - this is only the initial list, not even talking about advances in technology, to start the discussion - more later.

On bitter pills, hype-cycles, and event processing

My friend, Tim Bass, has decided to feed us with bitter pills about the state of the CEP market today - should we swallow ? on the other hand - The TIBCO CEP philosopher Paul Vincent is telling us that CEP is not a hype, and can eliminate the "fall" side of the hype cycle should we believe him?

To give you full disclosure - I don't take pills - sweet, bitter or otherwise, and let my immune system protecting me, without help -- maybe with age I'll have to desert this policy, but not out of free will. I also think that while CEP is real, there is some hype phenomenon around it - and the hype cycle is a very smart idea - representing the laws of nature.

So - a few subjective assertions about the state of the event processing market:

1. Event Processing applications are real, there are cases in which they provide a significant value (and ROI) for customers, for various reasons (still owe you postings about the different ROI I found).
2. Even the non-visionary spreadsheet-oriented vendors identify revenues (or losses if they'll not jump into the CEP wagon), thus, it seems that the size of the market starting to hit a critical size. The list of "reference customers" is some indication, but does not provide a good one about the state of the market - some small companies go to the press with any signing on customer agreement to do pilot, while in bigger companies only mega-deals are reported this way; besides there are customers who are not ready to get public exposing even in titles what they are doing.
3. We are still in the first generation of technologies - and first generations are typically the prelude, the thinking about an area is changing, and requirements are identified much faster than the ability of vendors to cope with them (so they start with hacking around them). To provide an analogy - we are now in a similar situation to the database technologies in the early 1960-ies, before the relational database has been invented, even before DBTG proposal. There were some products - but their utilization was limited.
4. While there is a real value to customers, there are also some of the hype phenomenon around it, which leads to the "hype cycle" as a predictor --- we are still in the climbing side of the hype curve - but at some point we'll see some more understanding around what EP applications may do -- and those vendors who will not be able to cope with the relative rapid changes that will be required -- will stay out of the picture, which is also a normal things. We have today more CEP vendors than DB vendors, so we are not yet in steady state.
5. Tim Bass complains about looking at the history -- saying we should focus on applications and history has no importance -- Contrary to that, and looking at other disciplines, there need to be a standard basis of the discipline, otherwise it become a collection of ad-hoc efforts that don't reuse the type of thinking. Standard thinking comes from unified theory - and we have not found yet the equivalent of the "relational model" in databases. In order to do it we need to go back to the basics in various disciplines, and see what can be pulled together - thus the work like David Luckham's survey about the history and the contributing disciplines is very important to get people from these disciplines to work together - like the Dagstuhl seminar or the upcoming DEBS conference

So what is the bottom line of all these assertions ---

first - I am optimistic, I view EP as a disrupting technology for the computing industry, and it has an important role in architectures, applications and conceptions of enterprise computing.

second -- the first generation is just the start, a lot of work to be done to get to the "hype cycle plateau" and beyond...

Since one of my declared hats is a catalyst - I prefer to use a constructive tone -- saying -- here is what should be done in order to get there, and also put a modest contribution to make it happen -- is more constructive from swallowing pills. In one of my next postings - I'll continue this postings by providing views on "what are the most important things to be done".
more - later.

On Event Processing Web Sites

In the last few days there is a debate in the community around the launch of Marco's wiki entitled "event processing wiki" - one opinion said - we already have a portal - David Luckham's site that contains a discussion forum and there will be more visibility to postings on a more known site. The other opinion is "let a thousands flowers bloom" an indication for the vitality of a community is in multiple sites, wikis, blogs etc.

You may ask yourself - what is the Rabbi's picture doing here? do I think that this is a religious question, or have I become religious in my old age. The question is - none of the above - this discussion reminded me an old Jewish story (nothing to do with Rabbi Malkior, an ex-minister and member of the Israeli parlaiment, whose picture you can see) :

An husband and wife came in front of the Rabbi so that he can judge in a dispute among them. The Rabbi listens to the wife and says - you are right, then the husband told the Rabbi his claimes and the Rabbi said - you are right too. After they have gone away - the Rabbi's assistant asks him - how is it possible that they both have been right, they told you contradicting things - and the Rabbi answered patiently - you are also right, of course.

This, more or less, reflects my opinion on the discussion. I totally agree that we, as a community, should not have a central control, and communities are distributed by nature and not centralized. I also agree that posting on a known an popular portal has more visibility then on not-yet-established one, so I'll leave it to the "market forces".

Talking about portals - there is another interesting EP portal now, dedicated mostly to scientific papers in this discipline, and include references to many articles: event based.org

Bottom line: At least cross-reference...

On Killer Applications

My friend Tim Bass, the popular blogger, is the person who implored me into writing my own Blog again and again, until I decided to give it a try. Tim, whom I always enjoy to hear talking, even when we agree to disagree, has notified us in his Blog that he is going to participate in a Webinar about "BAM as a killer application for CEP" . While the Webinar has not yet occurred, and I don't have a clue what they are going to say - I would like to dedicate this posting to killer applications and EP, since this is not the first time that I've heard assertions about killer applications. To start the discussion, let's look at three assertions:

1. BI is a killer application of databases
2. Electronic commerce is a killer application of the World Wide Web
3. BAM is a killer application of CEP

Before looking at assertion number 3, let's look at the previous two assertions:

• Databases are used for many purposed, certainly BI uses databases, which implement data warehouses, but most of the uses in databases today are on operational systems, master data management etc - and BI is just one application. The opposite holds, one cannot do BI, without storing historical data, thus databases is a killer technology for BI.
• Electronic commerce is certainly a growing area, and maybe there are people whose main usage of the web is electronic selling or buying, but today I have entered the Internet several times, none of them has been in order to buy. Actually, my own buying from the internet (I am buying books and songs - as I listen to music in the background when I am not in a meeting, so I have a collection of around 1500 songs now - and growing - only legal downloads!) issue a very small part of my use of the Internet, this is true for most persons, however, the converse holds - it is difficult to hold electronic commerce without the Web, so the Web is a killer technology for EC.

Now, back to the assertion that we inspect - "CEP is a killer application of BAM" - this would have been true if most of the CEP usages were BAM applications, Wikipedia description of BAM states: The goals of Business Activity Monitoring are to provide real time information about the status and results of various operations, processes, and transactions

Many of the BAM products concentrate around displaying Key Performance Indicators on dashboards, but even if we extend the notion of BAM, it is still observation on operations according to predefined measurements (typically aggregative ones).

Now the question - whether most CEP users are doing it through BAM ? according to my observation on the CEP market, the answer is - NO. BAM is an important application of CEP, but there are others - the early adopter application - algorithmic trading - is not really BAM - it is more RTE type - it makes decisions and not presents observations, system and network management applications are also mostly not BAM - they are diagnosis, attempting to find root cause for problems and not display measurements, and the same is true for information dissemination systems that don't monitor anything, and predictive systems that don't have measurements. Thinking about a sample of CEP applications I have looked at recently, there is certainly some that are of BAM type, but it is not the majority. CEP is being used for different purposes, and has different ROI to different people, I'll write more on the different ROI's in one of the next postings, thus, like databases, it does not have a single killer application.

The interesting question is if CEP is a killer technology for BAM or any other application type ? but - I have written enough for today. BTW - speaking about BAM, I have noted the posting of James Taylor - "why are enterprise application so dumb?" - doubting the benefits of presenting observation to humans, instead of taking automated decisions - food for thought. More Later

On Event Processing and Now.

After some more technical postings, back to macro-level issue. David Luckham has written recently in his website about the history of CEP (I'll refer to the content of this article in another posting). All indications are the event processing is not a new thing, however, as some people indicate, there are a lot of interest, events, maybe hype around it NOW - the question is what is new ?



The first observation is that unlike the past, today there are commercial "on the shelf" products whose main purpose is to provide event processing capabilities, while in the past there were event processing capabilities in other types of products (simulation, databases, network and system management, middleware, real-time systems etc..), there is also a start of event processing as a discipline. What are the reasons for the interest now - some of what happened in the last few years that supported the shift from hard-coded event processing functionality to COTS are (based on discussions with people in multiple industries):

• Some contemporary applications which are event-driven by nature - such as: compliance with regulations, the need to detect frauds as two examples - have become pervasive in multiple industries
• The increasing complexity of inter-process integration that is simplified with event-driven interaction
• The need for flexibility and agility to gain market advantage in different areas - thus, move away from hard-coded solutions.
• The substantial growth in the number of available events - e.g. since RFID technology became pervasive.
• Some industry trends like - BAM, RTE, "on demand" - that are also based on responding to events.
• The drive to save expenses in back offices by automating exception handling - trends like STP.

I am sure that there are more of these.

Thus, while event processing functionality is not new - "event processing" as a first class citizen in the computing world - with its own dedicated products, community and emerging discipline is new. More - Later