EPTS event processing glossary 2.0 is now available

The event processing area keeps evolving and there are more terms being used.  The EPTS glossary workgroup, co-chaired by Roy Schulte and David Luckham has published a new version of the EPTS glossary.  It  contains both old and new terms.    

On interactions between transaction and events

As a follow up to the transactions vs. events posting,  I would like to discuss some interactions between event processing and events.  ACID  (Atomicity, Consistency, Isolation, Durability) are the properties of classic transaction systems that guarantee serializibility. 


 Some people think that event processing and transactions live on two separate universes,  as event driven architecture advocates asynchronous mode and decoupling.   Note that there are transactional asynchronous systems (e.g. messaging with guaranteed delivery).    


One of my past Master students, Boris Shulman, has investigated the issues of interactions between transactions and events. There was no paper published from his thesis,  he has done the thesis while working for IBM, but then went to work in a start-up and did not have time to complete a paper, and the topic was not on top of my priorities either,  but recently I am thinking about it more. 


In the thesis he pointed out several different motivations to look at the combinations of events and transactions:

1. A monitoring system that monitors a transaction system,  events are emitted from the running transaction systems,  in a way, emitting such events before transaction commit, violates the "isolation" requirement of the ACID.   However,  at some cases there are timing constraints, since the monitored event need to be processed without delay.   This gets complicated when the transaction aborts, and all transactions operations rollback.   In this case the monitored event conceptually never happened. 
2. Event processing is done as part of an actual transaction,   the producer and consumer are part of a transaction, and the event processing is doing some processing that happen to fall into the functionality it support, but it is part of transaction.   Note that this does not view event processing as a stand-alone functionality, but as a functionality that can be embedded in other systems that can be transactional.  In the past when we did the Amit nodes inside IBM Websphere Message Broker, it worked exactly in that mode,  event processing as a specific node within a transactional message flow.
3. The event processing network (or a certain path in it) may have a requirement to behave as a single unit of work.   This is true when an EPA reads/write data, but also the raw events and derived events may issue transactional behavior.     

There might be other cases,  but these cases indicate some cases for interactions between transactions and events.   The transactions that relate to events, may not be  pure ACID transactions, I'll discuss other model of transactions that might be appropriate - later. 

Transactions and events: old world vs. new world

Paul Vincent posted on the TIBCO Blog some thoughts on transactions and events.  His story was mainly about bookings in a sleeping train that should have been in the same cabin, but the system did not have any way to do it in a single transaction,  In DEBS 2011 one of the keynote speakers Johannes Gehrke talked about similar things, of not being able to coordinate travel reservation of two co-workers who want to fly together (but still want to have independent reservations for the rest of the trip).   This is a symptom of the "old world" vs. "new world" phenomenon.    The "old world" in IT is the world of OLTP, which is centered around ACID transactions (maintaining: atomicity, consistency, isolation and durability).   The "new world" is a world of mobile apps, social media, NOSQL, Web 2.0 and more - this world has started from scratch,  the "cool" geeks who created it don't want anything to do with the nerds that work in the old world (slang I got from watching American TV programs, starting from "happy days"),    Anyway -- it seems that some relations do exist between the old and new worlds:  

1. New world applications should use some old world techniques like transactions, but this is not necessarily ACID transactions.
2. New world applications should sometimes talk with old world applications
3. Old world applications should support some of the new world characteristics -- and have to evolve.

All these three relations do exist, I'll write in the future about each of these topics.    


Last but not least -- back to the title of transactions vs. events.   Event processing seem to have some of the characteristics of the new world, however, there are some synergies again between transactions and events. 
One of my earliest posts on this blog, in 2007 had the title: "Event processing and transactions - real, real time, and real time enterprise", where I discussed briefly the idea of event processing within transactional system.   Since then there was some progress in the thinking - I need to revisit this issue on the blog.  
So made some promises on future posts on this Blog - and now back to the new world of  looking at what is new on Twitter, Facebook and LinkedIn...   More -later