On VLDB 2010 -- events and streams related papers

After a few years of missing VLDB, I plan this year to participate in the VLDB conference in Singapore (which will be an opportunity to visit Singapore, I have never been there). I have a tutorial accepted entitled: Event Processing - past, present, future. VLDB is one of the major research conferences of the database community (my original home community).

The list of accepted papers is now on the website -- looking at it there are some papers whose title include either events or streams:

Complex Event Detection at Wire Speed with FPGAs

High-Performance Dynamic Pattern Matching over Disordered Streams

Achieving High Output Quality under Limited Resources through Structure-based Spilling in XML Streams

SECRET: A Model for Analysis of the Execution Semantics of Stream Processing Systems

Recognizing Patterns in Streams with Imprecise Timestamps

On Dense Pattern Mining in Graph Streams

Database-support for Continuous Prediction Queries over Streaming Data

Conditioning and Aggregating Uncertain Data Streams: Going Beyond Expectations

From a Stream of Relational Queries to Distributed Stream Processing

iFlow: An Approach for Fast and Reliable Internet-Scale Stream Processing Utilizing Detouring and Replication

And some demos:

Active Complex Event Processing: Applications in RealTime Health Care

Efficient Event Processing through Reconfigurable Hardware for Algorithmic Trading

Geospatial Stream Query Processing using Microsoft SQL Server StreamInsight

As you can notice - in the database community the term "streams" is more common than the term "events", and I'll go back to the discussion of streams vs. events soon.

IBM will have a substantial presence in VLDB with 9 research papers, 5 industrial papers, 3 demos and 1 tutorial.

More - later.

On restful event producers

Another year has passed and yesterday was my birthday; I tend not to celebrate an event that reminds me that I am getting older, so I found on the Web a greeting card that fits my sentiments; however -- this year I got much more birthday greetings than usual. Of course I can be encouraged that all of a sudden I became popular, but the truth is that I filled profile descriptions in some social networks - Facebook, LinkedIn, the family tree site (may be one or two more) - all of them send alerts to people about birthdays...

I have written about event producers such as the CICS event producer, that instruments CICS transactions and emits them as events; event producer needs to produce the event by sensing or instrumenting some software/process to create the event, and also needs to communicate the event to the event targets (event processing agents, channels and consumers).

There is a new article on IBM developerWorks dealing with emitting events using REST interface. The word REST has an association with resting, like the Van Gogh famous picture of noon rest - below.

But in this context REST of course means "representational state transfer". The article describe how to use the REST interface of Websphere Business Events to emit events from any application that has access to web protocols. Enjoy!

On event flow and control flow

Teaching another internal course in event processing, one of the questions that was asked by the participants, and is actually a frequently asked question when viewing our notation of event processing network is -- whether we are reinventing workflows.

The answer is -- no.

Viewing control flow and event processing networks (both figures were taken from IBM developerWorks)

The common denominator is that both of them looks as graphs with nodes and edges of various types; but there are also some differences:

In control flow systems the edge typically means that when some task that executes on the node ends the control (conditionally) is passed to the task represented by the predecessor node in the graph. There can be some and/or/xor conditions as seen in the diagram above - and they typically refer to the termination status of the task (success/fail).

In event processing network, the nodes represent EPA (event processing agent), and the edges represent events that flow between these agents. The fact that an EPA emits an event does not necessarily mean that it has been terminated, an EPA can create derived event in an "immediate mode" which might designate that a pattern was matched. The source EPA may still continue to run. Likewise, an event that is sent to a target EPA does not necessarily mean that it is invoked, it may have been there, it does not even indicate that the target EPA will use this event, it may filter it out on entry. If it uses the event, it may not react to it directly, since it is a part of a pattern that has not been completed yet. In general -- all an edge in the graph says that one node sends an event to another, and nothing more.

One cannot represent event processing network using regular workflow, however, a workflow can be represented using event processing network !