On event flow and control flow

Teaching another internal course in event processing, one of the questions that was asked by the participants, and is actually a frequently asked question when viewing our notation of event processing network is -- whether we are reinventing workflows.

The answer is -- no.

Viewing control flow and event processing networks (both figures were taken from IBM developerWorks)

The common denominator is that both of them looks as graphs with nodes and edges of various types; but there are also some differences:

In control flow systems the edge typically means that when some task that executes on the node ends the control (conditionally) is passed to the task represented by the predecessor node in the graph. There can be some and/or/xor conditions as seen in the diagram above - and they typically refer to the termination status of the task (success/fail).

In event processing network, the nodes represent EPA (event processing agent), and the edges represent events that flow between these agents. The fact that an EPA emits an event does not necessarily mean that it has been terminated, an EPA can create derived event in an "immediate mode" which might designate that a pattern was matched. The source EPA may still continue to run. Likewise, an event that is sent to a target EPA does not necessarily mean that it is invoked, it may have been there, it does not even indicate that the target EPA will use this event, it may filter it out on entry. If it uses the event, it may not react to it directly, since it is a part of a pattern that has not been completed yet. In general -- all an edge in the graph says that one node sends an event to another, and nothing more.

One cannot represent event processing network using regular workflow, however, a workflow can be represented using event processing network !

On our proposed EU project - Take I

CDG Airport, Paris.

This is, more or less, how the A3 road in Paris has looked like today when I was on my way to CDG airport, when I asked the taxi driver how long it will take to the airport he said - no more than 40 minutes, well it took 2 hours, and could have taken more, unless at some point he lost his patient, drove through the next exit, and navigated with his GPS through some small streets
to get to the airport from a different direction. Since the price of the taxi is a function not only of distance but of time also, then the price also exceeded (by far) his estimate... This was not the only traffic related event. My airline sent me SMS to notify me that my flight home has been cancelled and they re-booked me on the next flight 2 hours later (that's why I was not pressed of the long trip to the airport). They sent me the SMS in French, while I was sitting in a train, and the passenger sitting opposite to me was kind enough to translate me, 5 minutes later they re-sent me the same SMS, this time in Hebrew.

I ended the day in visiting ILOG, acquired by IBM, and before that in the morning, together with some of our partners, we had a meeting with the EU commission in Brussels, about a proposed EU project. This is planned to be a relatively big project that will look at the future web services infrastructure which will be event-driven, and include event-enabled BPM. . We got good and useful feedback, and have a lot of action items, like finalizing the consortium. I'll write more about it in the next few months.

On event Driven BPEL

WS-BPEL has become a leading standard in the area of business process management, and since event processing is typically part of a bigger picture, it often interacts with business process management systems. One of the success factors is to make the environment ready for event processing, i.e. being able to get the right events, and being able to perform the right actions, while the event processing network has detected a situation (in one of the next postings I'll discuss how situation is related to the EPN world).

My student, Alex Kofman, has been doing recently a M.Sc. thesis looking from the BPEL perspective, he has determined what need to be done in order to extend BPEL to be "event driven", and proposed detailed modifications to the standard. This is intended to enable easy interaction between event processing system and BPM systems that plays as producer and consumer from the event processing viewpoint, while from the BPM viewpoint, input events can serve in decisions about orchestration, while output events can provide decisions external to the BPM based on the BPM state.

Here are some of the main points raised in this thesis:

1. Ability of BPM subscribe/unsubscribe to events.
2. Ability of BPM to publish events.
3. Ability to invalidate one or more processes as a result of an event (partially exists).
4. Ability to start a new process as a result of an event (partially exists)
5. Ability to invoke a task upon occurrence of an event
6. Adapt process execution upon occurrence of an event
7. Support the definition of an event processing pattern requested by the BPM engine, as a "callback" to an event processing system.
This in general will raise the level of abstraction and make the integration easier...

More about this project, later -- the thesis is now being written, Alex gave presentation about it last week in the department seminar, so this is a good time to start communicating these ideas.

CITT BAM-BPM-CEP-EDA conference Day 1

This is another picture from Regensburg - a picturesque medieval town on the Donau. Today I have attended the first day of the CITT conference organized and navigated by my good friend Rainer von Ammon .

The conference is held in an old historical building, and like the VLDB that was held in a historical building in Vienna, the architect of that building also is in the opinion that people love going up and down the stairs multiple times... Anyway -- I have learned several interesting things - one that is not really related to the conference-- one of the speakers revealed that there is a formula to calculate the value of any digit in pi without calculating the other digits - and indeed after some search in the internet (wikipedia does not seem to know about it) I have found it .

Other than that - some discussion about creating a consortium for an EP European project, some presentations about "fraud detection", a keynote address was given by Stefan Reid from Forrester Research - the topic has been - " ESB Open Source Challenge", and he discussed what is ESB and is it mature enough to be an "open source" topic. His ESB definition has been too broad to my taste (i.e. he sees business process modeling and monitoring as part of ESB, I think that they are not really ESB functions, and architecturally they are in another category). He also said that "in the end of 2008 any ESB that will not include CEP will be obsolete" - and I obviously like this statement. I think that ESB (in the SOA case) is the major (but not the only) carrier of CEP (but I'll discuss ESB in a seperate posting). Later in the day Rainer brought his entire class of students and they presented BPM oriented projects. The last presentation was done via Skype from Redwood City - two of Rainer's students that are doing a project in Oracle has presented their project, while the two CEP Oracle champions - Dieter Gawlick and Shailendra Mishra (my EPTS partners), have presented the upcoming Oracle CEP. Well - there is some room to improve the skype technology.... There was also in the middle a panel about BPM patterns - and somebody made the observation that people talk about technical patterns, while there is no enough work on business patterns - this is true in a way for EP as well, but I'll discuss patterns in another posting. Tomorrow - second day, where I'll have to earn my trip by giving my keynote talk entitled: "event processing as part of enterprise architecture" - well, since I am working in a "just in time" mode - I'd better finish this presentation.

PS.

Tim Bass complained in his popular Blog about the lack of possibility to read blogs and from China -- I have checked in the log of my blog (thanks to google analytics) -- it seems that there are (not many but some) people form China that are reading my blog - I even have one reader from Xiamen (most are from Beijing), so probably there is a way to overcome it. I have done a business trip to China (Beijing only) in 2004, and it has been very interesting experience, indeed when I watched CNN in my hotel, and they started to talk about demonstrations for democracy in Hong-Kong, there has been a sudden communication obstacle that has been resolved three minutes later, but this has probably been a conicidence.

I am also glad for Tim that democracy has been restored in Thailand after some period of military control.... things are fragile in our universe... Back to my presentation now - more later.

On Trifecta and Event Processing

Reading the educated blog of my friend Tim Bass
I had to admit that I don't have any clue what a "Trifecta" is, and rushed to wikipedia

for help to find out that in horse races - it is a bet on the first three horses. I must confess to my ignorance - I have never watched a horse race, neither have I gambled on horses. I have made a major technological gamble though, around 10 years ago, to bet around "complex event processing" (without using this name and without knowing on the three others who have worked in parallel on the same topics - will have at some point postings about the history), and after suffering some frustration periods, it seems that the reality also slowly getting there, and I still hope it will be the "next big thing" - but we have some work to do in order to get there -- again, a topic for another posting. Now the question posed both by Tim Bass, and by Joe McKendrick

is - whether CEP is depended on SOA, and in turn - does it require the coupling of SOA and BPM - which is by itself a problematic concept.

My answer is -- CEP is an horse that is indepdent in any other horse, donkey, mule or tiger. As I noted in earlier postings, SOA and EDA are orthogonal - SOA is about modularization or componentization of the enterprise, and hence it IT systems, introducing modules as "services". Now let's talk about the relations between SOA, CEP and BPM.

(1). Services in SOA can be producers and consumers for "Event Processing Agents"; as a producer - the service is instrumented to emit events, as a consumer it can be notified or triggered by an event; Services are one type of consumers and producers but not the only type - every application can be consumer or producer, thus there is a loose coupling.

(2). In SOA environment there is an ESB (Enterprise Service Bus), whose functions have some overlap with event processing (especially mediated event processing) - can be a natural host for the "event processing network" (I'll write more on ESBs in another posting). However, again Event Processing Agents can also run outside ESB.

(3). Workflows (BPM) are special type of consumers and producers. In the SOA world, BPM orchestrate services, and thus, BPM can emit events about the status of the workflow, while as a consumer it can add/modify/delete one or more workflow instances. Some of the Event Processing applications (e.g. some types of BAM applications) are based on business processing - however, even in SOA environment this is only one type of consumer and producer, and there are other types (e.g. RTE applications which process events in-line and not in observation mode).

Bottom line --- Event Processing can have different interactions with SOA, and when IBM's announcements in this area will be available you'll realize that there are different entry points. Event processing can also work in legacy and non-SOA environment.

So today I have learned about horse races and promised some more postings (not promising when) -- more, later.