Tuesday, September 2, 2008

On flow oriented and component oriented development of EP applications

I got an invitation for some company I have not heard about for a kickoff of a product in the area of "seating in front of computers", I don't have time to go - but since they have also attached a picture of their product, I have copied it - I wonder if this is the current trend in ergonomy...
Anyway, today, some thoughts following a recent discussions here about development tools for event processing applications.
There are two possible ways, from the developer's point of view, to build an event processing network.
  • The first, which I'll call "component oriented" (may be there is a better name?), in which the developer defines the different components (patterns, rules, queries - use your favorite language style), each of them individually, and then some compilation process build the network implicitly. This is a kind of "bottom up" approach.
  • The second, which I'll call "flow oriented" , in which the developer has some graphical representation of the flow, and when building the flow, one can put some boxes inside the flow, and zoom to each box to define the component. This is a kind of "top down" approach.

It seems that each of them has benefits for other assumptions - if the application is dynamic and mainly subscription based, then the first approach is probably better, since the notion of flow is not a stable one; if the application is relatively static, then there is a benefit to use the second approach, since it can provide more visibility into what the application as a whole is doing, and help in the validation, since the "decoupling" principle may bring to the developments' feeling of chaos (this is indeed one of the barriers of the use of event processing...). As said, the "flow oriented" approach can ease the validation of event processing application, there are also some tools that help validating "implicit" flow, but the validation issue deserves discussion by its own right. More - later.


Hans said...

Could you give examples of each kind? I'm not sure that what your talking about really has much to to with graphical flow vs. non-graphical components.

To relate this to two techniques that I have used:

one technique uses components that talk to each other via strongly typed interfaces - this is often represented in UML as sequence or collaboration diagrams.

The other technique involves components that have very weakly typed interfaces and essentially have an in and an out port that take and send some very general data structure. These components are configured and wired together in some way, maybe graphical maybe in a text file (see the Streams component of the ACE C++ framework for an example of configuring stream processing using loosely coupled processing agents).

Opher Etzion said...

Hi Hans. I usually don't refer to specific products, but I'll do it now to give you concrete examples -- (as I have not looked at these products recently I apologize if I am wrong !).

Type 1 - component based - look at Rulecore - in Rulecore there is a notion of rule, which is the component, each rule can created derived event (AKA situation) which is an event, the event is going back to the "event cloud" and can be picked up by other rules as an input. The flow in this case is implicit.

Type 2 - in Streambase there is a notion of "flow" in which, in the interface level, one can drag and drop an operation type as a "node" in the flow, using this GUI, the flow is explicit.

Of course, there are other products for each of the types, these are the first two that come in mind...