Dagstuhl seminar on event processing - the first day

It is evening here in Schloss Dagstuhl, the castle that you can see in the picture above. My room is on the ground floor of the part of the building in the left-hand side of the picture. This seminar organized by Rainer von Ammon, Mani Chandy and myself, is a five days event, and we just finished the first one, during the day I lost my voice for about an hour, and regained it by eating ice-cream. In this seminar we are working in five sessions, you can see this presentation includes the outline of what we are doing in the opening session and the one slide that every participant (well - almost, a few have not sent the slides) put about himself/herself.

Besides the opening session we have done the first meeting of each of the five groups, and had initial discussion about them. Tomorrow -- two breakout sessions and two deep dives. One of the things we started to discuss and we'll finish this discussion later this week is ---- looking in the future five years from now - how can we measure the success of this seminar to change the world --- more about it later.

SAP acquired Sybase: the event processing angle

It seems that Paul Vincent will have to update his product genealogy again. There are various analysts like James Kobielus or Philip Howard have expressed their opinion about the motivation and possible implications of this merge, and I leave this discussion to them; I'll write on the narrow perspective of the event processing angle (to justify the name of this Blog).

Taking the EP perspective, there is a series of big fish swallows smaller fish, until it gets to the ultimate big fish. Event processing is shifting from being dominant by start-ups to be dominant by bigger companies, which happened before in other areas, and is sign of getting this technology to the main stream of computing, this is consistent with some of the trends we identified as the current trends in the conclusion chapter of the EPIA book, event processing is going from narrow domains to wider domains and from being stand-alone technology to being part of bigger frameworks. As four big fish -- now the four big software companies - IBM, Microsoft, Oracle and SAP are all in the EP market, and all of them view EP as part of a bigger game; I guess that there will always be smaller stand-alone EP companies for niche markets, but most applications do not live in an isolation and have some relations to other areas - BPM, BRMS, BI and some more.

The fact that most players in this market are big and medium (TIBCO, Progress Software, Software AG) companies, can make the climate more comfortable to advance standards now -- which is one of the topics we plan to discuss in Dagstuhl next week.

When we established EPTS, SAP was not interested to join, since they did not have a product at that time, we'll see whether they will want to inherit Sybase's membership (which by itself was recently inherited from Aleri)... We'll stay tuned for more on this front...

Some footnotes to Brenda Michelson's post on TUCON

Las Vegas is a place where companies like to do customer conferences; I guess that the quantity and sizes of hotels is a major factor. Last week IBM has done the big customer conference: IMPACT 2010, after a few years of attending this conference, I have skipped this one, so I did not cover it in the Blog. This week TIBCO has done its customer conference - TUCON 2010, naturally I have not been there either, as I am not a TIBCO user, but looked at some impressions on the Blogland, and came across Brenda Michelson's Blog describing the keynote sessions. With the disclaimer that my information on this is second hand (but a good hand !), I'll make some footnotes:

1. I believe in the direction that events will be more and more in the center of enterprise (and other applications); we see it happening and we'll see much more.

2. Like Brenda, I am not sure that the phrase "enterprise 3.0" will catch, at the past there was an attempt to push "SOA 2.0" but it did not catch either, I think the world became tired from recycling the n.0 phrase.

3. "2 second advantage" is catchy, and may be true for some applications, but in some of them 2 second might not be enough to make a difference. In general there are two aspects: immediate reaction -- react fast (or faster than others); and proactive --- react in time to mitigate or eliminate future event; I am not sure it that if the Interpol will know about a crime 2 seconds earlier it will be enough time to eliminate the crime.

4. About NoSQL and relations to event processing - I'll write a separate post, it is an interesting insight that NoSQL is gaining traction.

5. Context as additional dimension -- I have written several times in the past about contexts, and am planned to deliver a tutorial about contexts in DEBS 2010; I believe that contexts are starting to be a major factor in computing.

It is good to see that the event-centric universe is getting more traction. More -later.

On inter-system event causalities

A lot has been spoken and written about the Wall Street Plunge, David Luckham raised the question about the role of event processing systems in prevention and/or postmortem analysis of understanding how such a situation happened. I am not sure exactly how much event processing system have been part of the reason of causing this plunge in the first place, but here we have an interesting challenge, the challenge is that we need to understand causality in a real event cloud, where the events are part of multiple systems. The nature of the challenge is twofold:

First: Each of the automated trading systems logic is, of course, not transparent, since they are used as competitive edge by the traders, thus it is difficult to know how various systems would function, and there will be a need to learn their logic (which can be constantly changed) based on their actual behavior;

Second: The logic of the various systems is expressed in various ways, and there is no standard causality model that can be used across systems.

This can be one of the challenges that we can put into the challenges of the next generation of event processing systems, that we plan to discuss in Dagstuhl.

EPIA in video

Today, the Manning marketing guy, Steven Hong, attracted my attention that there is a recent video on YouTube that talks about CEP and shows in the last slide an adevertisment to the EPIA book (now in production phases -- most chapters already in typesetting).

It is interesting to see on the side various videos that deal with EP, including IBM'S CICS related stuff.

This remind me also that I forgot to mention the recording of the Webinar we gave in March 2010 on the ebizQ site. You can watch the recording (may need registration to ebizQ).

Have fun.

On small vendors, big vendors - individual view and industry-wide view

If you want to have a pet animal, you might prefer the little kitten over the big dog; if yo want to have somebody to guard your house, a big dog might be more effective, there also people who prefer big dogs as pets, a matter of personal taste.

Marc Adler responded to my previous post about consolidation and pure play in the EP market. Since Marc is in blog reduction mode, I will not be insulted if he'll not react to this posting, but I would like to provide further perspective.

There is nothing in Marc's response that I don't agree with, in fact when I have been 30 years younger (having 30 kg less from today, and this is after I've reduced 21 kg in the recent year).

I have worked in the IT shop of the Israeli Air-Force, I have been working with a new product that was called DB/1 and later renamed as Sapiens, at that time positioned as application generator for data-driven programming (there has been evolution in positioning over the years as well), it was a product in its beginning, I have done the first project with this product on the premises of the product developers, went with them to lunches, and went to their family weddings and funerals, thus, I got excellent service for them, could ask to add features, when there was any bug or problem I know exactly who is the responsible developer and could call him, or even invite him to solve it in our site. This has worked well since I had good personal contacts with them, but moreover, at that time the number of customers they had could be counted on the fingers of a single hand, and they could provide much attention to every customer. As said, for me, it was the ideal product to work with, and I fought some of my colleagues and superiors who thought that this is a too big risk for the Air-Force to depend on them in critical application, which was of course true. I guess that Marc had somewhat similar experience with Coral8 in his previous work; I also feel sympathy to the claim that big corporates have an inclination to come to customers with more people than the customer expects to see, and has less intimate atmosphere with customers, this was always been true.

Fast forward 30 years, I have somewhat different perspective on the universe; it was somehow surprising for me to find myself being hired by a big corporate, from an employee's point of view there are pros and cons to be employed by a big corporate, there is a nice posting on this topic.

People whose small companies are acquired by big corporates sometime dislike the culture of big corporates and move on, sometime they adjust, I know stories of both types, it is not black or white.

My perspective now is more on the macro level and looking at the question of: Is the current wave of consolidation good or bad in general for the event processing area, its assimilation into main-stream computing, and the ability of play a significant role in current and future enterprise computing?

My own view is that the market moves to the right direction. As I believe that the larger market is not in stand-alone event processing applications, but as a pervasive technology embedded in enterprise computing in general, there is some benefit to companies like - IBM, Oracle, TIBCO, Sybase, Software AG, and now we also see that Progress Software is making event processing as part of a more general platform. Some applications may not need it, but many others do, and getting event processing in the mainstream enterprise software infrastructure, can be done through owners of such existing infrastructure.

One other potential benefit that I see is that with a market that is more dominated by bigger vendors there is stronger probability to get to standards. Standards is one of the signs of maturity for an area (e.g databases, web services), and will be vital to get event processing into the mainstream. We started to discuss standards in the pre-EPTS meetings in 2006, and at that time the dominant startup companies were very much against it, since they both did not see the value for themselves, and also feared that standards will distract their limited resources. Bigger companies have more standard oriented culture, and experience in other areas of how to do it right. I think that the current developments in the market provides a good opportunity to raise the standards issue again, and this will be one of the topics planned to be discussed in the upcoming Dagstuhl seminar on event processing. I'll write more about standards follows the conclusion of the Dagstuhl seminar.

Back to the original theme --- there are times in life in which one prefers small cats, and other times that one prefers big dogs. Small cats may be more cute and pleasant, but in this phase in my life I am going to hunt, and big dog will probably be more effective for that task.

On consolidation and pure play in the EP market

Marc Adler returned to the Blogland this week to claim, among other things that the fact that the list of pure-play event processing platform vendors is being reduced, is sad. The fact that there have been several acquisitions of "pure play" vendors recently is true, with the acquisition of Aleri by Sybase earlier this year, and the recent acquisition of RTM by Software AG. If you are interested in the genealogy, Paul Vincent is keep documenting it. I have traced some of the IBM acquisitions and realize that merges are sometimes tough from organizational culture point of view and sometimes there is a need to change direction in a not easy fashion, as shown in the picture above, so it might be sad for some individual people, but not necessarily bad from the industry point of view.

However, the fact that the pure play vendors are being acquired has another side to the coin, which means that the big or medium software vendors are buying them. This is a sign that event processing is getting to be part of the main stream of the enterprise computing, and is part of growing up of this area. This has also happened before in other areas, and was predicted several years ago by analysts to happen in the event processing area.

In fact, now most of the major players in enterprise software area : IBM, Microsoft, Oracle, TIBCO, Progress Software, Sybase and Software AG - have now an event processing platform as part of their enterprise computing offering.

This is an indication that all these vendors realize that event processing is required part of their enterprise applications, and that event processing is not really stand-alone but it is increasingly getting consumed as part of larger play, and getting to further areas and industries in addition to the early adopters of capital market trading applications.

There may still be a role for pure play event processing products, especially in various niches that are not being properly handled by the current products. Some of them may even develop event-driven enterprise computing platform, and join the medium companies, it happened before, but it is not easy.

While the bigger companies advance the projects at their own paste, the smaller companies as well as the research community sometimes have roles of catalysts to advance the area. We'll discuss the future of the event processing area in depth in the Dagstuhl seminar planned to start in 2 weeks. More about it - later.