Event processing as blasphemy

The fire in the Carmel mountain is still on  for the third day, although it is somewhat reduced due to the fire fighting aircraft from all over the world - the most notable one from Russia, but also from other countries; it is good to see that when a disaster happens, many countries in the world are getting to help, Israel has also a record of helping other countries while disaster occurs.   I guess that the fire will be overcome eventually, but this does not change the very bitter feeling that the population here has about the incompetency of the government, where a series of faults came together to bring this disaster.   For everybody who sent me worried Emails from all over the universe:  the fire did not get into the city of Haifa, so have not been in real danger, the site of IBM Haifa Research Lab, is relatively close to the fire area, as it is located on the Haifa University campus.  The campus was confiscated to be the headquarter of the fire fighting forces, so it is still blocked.   While in Haifa we were not in danger, people in some villages lost their home to the fire, and 42 people were killed when a bus was caught in the fire.   

Now for this posting's topic -- you probably wonder what's event processing has to do with blasphemy or religion at all?  I think that I have written before about all of these, but will put it within a single perspective. When I was young I had several friends who moved through the process of becoming religious (what the Christians call "borne again", Jewish people are using a different term), I watched this process with interest, and has many discussions with them (well, they tried to convince me that they saw the light and I just have to look more carefully to discover it).   One thing that I have learned about religious people is that it is useless to argue with them, since their beliefs are based on axioms, and once you identified this fact, one cannot argue over axioms, since this is the nature of an axiom.    Likewise, there are many professional religions, I have seen religious wars in other areas of computing, and this is not really a new phenomenon, just different gods.

Here are three religions for which event processing serve as blasphemy to that specific religions.

Religion one:   The data-centric religion.

The religion's belief:   the world is data centric, everything can be done within database tools. The is a small niche which requires high scalability, but it can also be dealt using database techniques,

The blasphemy:   events processing is a distinct discipline; it has some unique characteristics. 

Religion two:  The programming model religion

The religion's belief:   all functions need to be expressed using the programming languages we know and love.

The blasphemy:   event processing has various languages abstractions that are not part of the regular languages. 

Religion three:  The "true CEP" religion

The religion belief:  The term CEP was coined to cope with application of types of intrusion detection; any person who did not work directly on intrusion detection applications is not qualified as a priest for the religion, thus cannot really deal with event processing. 

The blasphemy:  Anybody using the term "CEP" for any other application type is a blasphemer, any technique that tries to address any other event processing application is simply irrelevant  (comment:  I don't really  tend to use the term CEP, but some of the vendors  indeed use it).

As said,  the prophets (and disciples) of these religions believe in them in an emotional way, and there is no use arguing with them, so the best way is just to expose the axioms they believe in and let people think whether they believe in these axioms or not.     One of the motivations of the EPTS use case survey is to find out about the  usage of event processing today;   since it is generally agreed that the event processing area barely scratched the surface of its potential, an equally important issue is to identify what are the gaps in th state of the art  that are required in order to achieve it, and this is another major activity of the community that will be discussed within the event processing manifesto and   other related activities -- more about these topics - later. 

Fire in the Carmel mountain

 The biggest fire in the history of Israel is now taking place on the Carmel Mountain, where I live.
It started in late morning, now it is evening and there are many branches of the fire. 

In the lunch break we went out for a short walk in the Haifa University campus, where our lab is located, and saw the sun looking like this - a result of  the fire. 

. 

This is another picture of the fire, taken from the Ynet site,  an Israeli Internet news site.  


The damage is huge, much of the Carmel forests have been burned, dozens of people killed, mostly those who came to evacuate other people.   Some villages and neighborhoods in nearby towns have been evacuated, and it seems that the local firefighters don't have means to control the fire, and have approached other countries for help, well - I guess the government has other priorities than to invest in firefighting equipment.    Meanwhile, where we are located it seems to be safe, but everybody is in the mercy of the wind directions...   more - later. 

More on ACM SIG

Some people who are not familiar with ACM asked me by Email (and one comment on the Blog) to elaborate on the idea of ACM SIG for event processing, what is ACM SIG, and what will be the benefits of creating one?


The main answer, IMHO, is to get recognition that "event processing" is a discipline, and to have the logistic support for publications (periodic newsletter and scientific journal) and conferences, ACM also manages the  budget for SIGs, whose income are memberships and conferences' revenues, a capability we don't have in EPTS.   Note that DEBS is already ACM conference, currently sponsored by two SIGs:  SIGSOFT (the software engineering SIG) and SIGMOD (the database SIG).  Complete list of current SIGs is on the ACM site.   This kind of recognition will help grow the research community in this area and can help in accelerating the development of this area.  The main benefit will be to the research community, whose members have initiated this idea.     


You can look at a recent proposal about establishing "social computing SIG" on the ACM site to get the flavor.

Towards an ACM SIG for Event Processing?

One of the ideas that have been discussed several times is to apply for ACM SIG (Special Interest Group) dedicated to event processing; the idea was discussed again last week in the EPTS steering committee monthly call.    The decision has been to apply to the EPTS members and the community at large and check the interest in devising this EP SIG.   This is not intended to replace EPTS but to have an established home for the research oriented activities of the communities.   In parallel to sending a call for interest expression to the EPTS members, I am also applying to the other readers of this blog;  if you wish to support this initiative, or think it is premature, or just a bad idea -- you can take the poll in the right-hand side bar of this blog.  

On the multi billion dollars, the accounting value and and the economic value

Came back today from a short vacation in the Grand Vista Boutique hotel in Kfar Yuval, at (almost)  the most northern (and east) part of Israel.    Looking at some related blogs, I found Joe Mckendrick's blog mirroring Colin Clark  saying that the "CEP vendors" failed to produce the multi billion dollars revenues they promised to their investors this is due to two facts:  1). end users cannot understand it; 2). it does not provide the IT people a tool to build a complete application.


I am sure that Colin, who has been a  key person in one of the first generation startups in this area is right in his perspective.   However, coming from the big blue, I have a slightly different perspective.  


I'll start from the last point, since this is a key to the rest:  the claim is that the "CEP vendors" did not offer a tool to build a complete application.  This is of course true; there are applications (indeed in capital markets but also in other industries) that have applications that are mostly pure event processing applications, however, this is not the major part of the market; the major part of the applications market is such that require event processing capabilities within an enterprise computing platform, that also includes BPM, BRMS, ESB and other stuff.  This is the approach taken by the bigger companies that got into the event processing game.  It is true that a start-up cannot do it alone, and start-ups that remain "pure event processing"  companies are confined to the niches of stand-alone EP applications (in which the potential was also not reached, so there is certainly a place for products in this niche).    As for the multi billion dollars,  I don't think that there is any analyst who predicted multi billion dollars for that market in 2010,  it takes time to develop a market, and as a stand alone EP products, may take longer time to get there.    When we did the business evaluation of the EP market around 3 years ago, we got to the conclusion that for the market of  "EP capabilities within an enterprise computing platform", one can look at accounting value or on economic value and they would yield different results.   Let me give a simple example in order to show what I am talking about:  let's say that a big enterprise is doing a software deal with the value of $15M for its enterprise computing infrastructure.   This may include many software components licenses.  Let say that the deal is a bundled deal, but if we'll try to isolate the value of the EP capabilities license, we'll get to  $500K.   However, in the RFP the event processing capabilities are required part, and a vendor that does not have the required capabilities is disqualified for this bid, then this vendor, losing the bid did not loose a $500K deal, but lost $15M deal.   This is the distinction between accounting value and economic value.   We see more and more RFP that either put event processing capabilities as mandatory, or give some weight to it as explicit criterion, this is done, even if the intended use of EP is planned in 2-3 years in the enterprise IT investment plan.     I think that looking at economic value (e.g. deals that have EP components inside) we might already passed the 1B dollars mark,  but this arithmetic, of course,  makes sense to vendors who own enterprise computing platforms and not to start-ups that has to earn the actual accounting value of stand-alone EP applications (or sell it as embedded solution).    About the "not appealing to end users"  - I'll write a separate posting about it; some of the analysis from the user's point of view was done as part of chapter 1 of the "event processing manifesto" (a follow-up to the Dagstuhl seminar on event processing) which is planned to be published in January 2011, and presented in March 2011 within the virtual EPTS symposium.   More about this topic - later.