Another type of proactive systems

Today I had a meeting with PhD student and her adviser to help them do sanity checks for a research idea related to event processing.    One of the ideas coming from them is actually looking at another type of proactive systems relative to what we have been looking at.   The difference stems from the fact that event producer can be of two types:  observer and creator.   An observer, e.g. sensor, observes that an event has happened and reports it, but it does not take any part in the event creation.   A creator is a producer that creates the event, and can decide whether an event is created, or in some cases can retract events.  Such a creator can be an instrumented program, BPM,  transaction system etc...

The proactive systems we have been looking at assumed that the producer is observer, thus we cannot control the creation of events but we can try to predict it, and eliminate it, or mitigate the damage it might inflict.   

Contrary, when the event producer is a creator, we can control the event.   There might be a few cases here.

One case can be that we again predict that event is going to happen since there is some causality relations with events that have already been detected, and within a certain context we don't want this event to happen,  we can then tell the producer -- if this event is being produced (again, within certain context, e.g. time window) -- don't produce the event.    Another case can be that the event is being created by a transaction system,  this event, within some context, and maybe in conjunction with other events, derives a situation that we want to eliminate.  The fact that the event is part of transaction that has not committed, gives the possibility to retract the event and eliminate it from being produced to the outside world when the transaction commits.  

The pure view of event processing puts producers and consumers outside the boundaries of the event processing system,  however, in reality there are cases where either the producers or the consumers (maybe even both) are controlled by the same logic, and thus detection of situation may impact the producers and consumers.   I think it gives rise to interesting applications.  I'll get back to this topic later.

On interactions between transaction and events

As a follow up to the transactions vs. events posting,  I would like to discuss some interactions between event processing and events.  ACID  (Atomicity, Consistency, Isolation, Durability) are the properties of classic transaction systems that guarantee serializibility. 


 Some people think that event processing and transactions live on two separate universes,  as event driven architecture advocates asynchronous mode and decoupling.   Note that there are transactional asynchronous systems (e.g. messaging with guaranteed delivery).    


One of my past Master students, Boris Shulman, has investigated the issues of interactions between transactions and events. There was no paper published from his thesis,  he has done the thesis while working for IBM, but then went to work in a start-up and did not have time to complete a paper, and the topic was not on top of my priorities either,  but recently I am thinking about it more. 


In the thesis he pointed out several different motivations to look at the combinations of events and transactions:

1. A monitoring system that monitors a transaction system,  events are emitted from the running transaction systems,  in a way, emitting such events before transaction commit, violates the "isolation" requirement of the ACID.   However,  at some cases there are timing constraints, since the monitored event need to be processed without delay.   This gets complicated when the transaction aborts, and all transactions operations rollback.   In this case the monitored event conceptually never happened. 
2. Event processing is done as part of an actual transaction,   the producer and consumer are part of a transaction, and the event processing is doing some processing that happen to fall into the functionality it support, but it is part of transaction.   Note that this does not view event processing as a stand-alone functionality, but as a functionality that can be embedded in other systems that can be transactional.  In the past when we did the Amit nodes inside IBM Websphere Message Broker, it worked exactly in that mode,  event processing as a specific node within a transactional message flow.
3. The event processing network (or a certain path in it) may have a requirement to behave as a single unit of work.   This is true when an EPA reads/write data, but also the raw events and derived events may issue transactional behavior.     

There might be other cases,  but these cases indicate some cases for interactions between transactions and events.   The transactions that relate to events, may not be  pure ACID transactions, I'll discuss other model of transactions that might be appropriate - later.