On Event Proessing Platforms and Engines

Here are pictures of some engine and some platform, and they don't seem to be compatible, however if a car will replace the horse in the 2nd picture, all of a sudden, the engine (of the car) will have a role in the platform. In event processing we hear more and more about platforms, and even about event-based middleware that will be a basis for XTP and other stuff. The platform is providing some services for agents to run, like a road system that can enable to get from place to place and provide services such as: signs, traffic lights etc... The way to go there can be by foot, riding an hoarse, and driving a car, among other options. Taking this analogy further -- in the event processing world, every agent can be implemented in ad-hoc fashion (going by foot), use C/Java code with some tools to help create EP applications, or use engines, which are COTS tools that do event processing. Implementation of heterogenous agents on the same platform requires interoperability standards, if we also wish to implement it in a seamless applications we'll need language standards. There are some starts of event processing platform, quite basic at that point, but the direction seems promising, and may, at some point in time we may see event processing platforms as a basis for the new generation of enterprise application servers.

On the envelope for CEP

In two recent blogs - Mark Tsimelzon from Coral8 argued for CEP server vs. embedded CEP libraries, while Paul Vincent from TIBCO argued for the need for infrastructure stack outside the

CEP engine.

Since I avoid making product evaluations in this Blog, I'll talk about the principle.

Conceptually we have a model of "event producer" that produces the events, "event consumer" that consumes the event, and the "event processor" which processes the events and stands in the middle. There are two questions about envelopes:

(1). Is "event processor" an embedded capabilities inside applications or an server.

(2). If "event processor" is a server, is it independent server or part of a larger middleware ?

The answer, as anything is not binary (zero or one), there are cases in which there is a need to have event processing as embedded capabilities (e.g. inside pervasive devices), but Mark is right that the big majority of "event processors" are tend towards the server.

The second question is more interesting --- we have today both stand-alone server and servers that are part of a larger middleware, the rational behind being part of a larger middleware, stems from the fact that event processing is not isolated and has various relations with different applications in the enterprise, it is true that the loose-coupling nature of event driven architectures eases the task of separating it from the applications, but still the integration is the most costly part of building event processing applications, and means to ease the integration has already built into application integration middleware, and if the event processing server is a stand-alone one, there is a need to re-invent this integration, as Paul Vincent rightly say: every $ a CEP vendor spends on middleware integration is a $ less on interesting CEP functionality.

Furthermore, there are some event processing infrastructure and functions (pub/sub, routing - for simple event processing, and ESB mediations like - enrichment, transformation etc, that are already there). Thus, it seems that the ROI will be higher if event processors will be implemented on top of a "middleware stack".

An interesting observation is that from the point of view of application integration middlewares - event processing is becoming a key feature, and there are already some predictions that the standard event processing programming model (which is still not there!), will be the basis for application servers of the future, e.g. Gartner's XTP that I have once discussed and should discuss more.

On CoDA - Context-Driven Architecture

Yefim Natis from Gartner, the same analyst who is bringing us the XTP vision
is also responsible for the brand new vision CoDA = Context-Driven Architecture; which is one or two steps further from the XTP era. In the previous posts about contexts: (1) ; (2)
I have argued about the use of context as a first-class-citizen in event processing; and indeed people act and think within contexts, and the use of explicit context can get the computing closer to the human thinking, and also has computational benefits (like indexed vs. sequential search in the early days of file systems). Both Yefim and myself did not provide formal definition of context when talking about it, and I still have a "to do" to think about it. One comment though about the "Architecture" - some people think that SOA and EDA are contradicting terms, since there can be only one "big A" - meaning architecture. As I have already mentioned - in the posting about EDA and SOA that they represent different aspects of computing, thus they are orthogonal (but can co-exist). The question is - whether context is yet another dimension that can co-exist with the other aspects ? again - still need to think about it. Thus, I'll return to the context concept soon.

XTP, Distributed Event Processing, and Distruptive Technologies

Hello everybody. Another experience of novice blogger - I had to start moderating comments, since I got spam attack of comments - so comments will not be seen in real-time anymore - but keep sending them anyway :-)

I have heard couple of times in the last few months talks from Yefim Natis
from Gartner who promotes now the concept of XTP - "Extreme Transaction Processing"
Yefim claims that XTP is a disruptive technology, in the way that it will be a new wave in enterprise application servers, he mentions in particular two disruptive technologies: dynamic grid architectures and event-based programming models. My friend Alan Lundberg asks in a recent blog what is the difference between CEP and XTP -- the short answer is that XTP is a framework that should have CEP capabilities but some other capabilities such as: transaction management, highly distributed environment etc... It is reasonable to think that some CEP applications will run on XTP, and maybe for some it will be overkill, but this depends on the ability of XTP to scale down as well as scale up. Another friend, Tim Bass (the person who talked me into writing this blog), has also recent blog about distributed event processing in which he said that "CEP engines should operate as cooperating intelligent agents" - and this assertion is the essence of the "event-based programming model" for XTP. I'll drill down into it in later blogs, but just point out some of the principles:

• In the event-based programming model - events and their flow are the most basic primitive
• Events are flowing among agents - agent may be intelligent agents that are doing "smart" things, or just software agents doing predetermined computations.
• The wisdom of the system is both in the agents and in the network - the network itself is intelligent and getting decisions.

This is another view of the world than the current programming models that are based around objects (and recently services), and may act as a disruptive programming model towards them, since having event as a first class citizen changes things.... These are preliminary ideas, but I find this idea intriguing and fascinating, so I'll probably dig into it deeper.

More - Later.