More on the nodes in the event processing network

This is the illustration of the event processing network for the Fast Flower Delivery example that accompanies the EPIA book we are writing, don't spoil your eyes reading it. We'll provide the readers with an open source editor to the EPN meta-language we have developed (developed by Ronen Vaisenberg, a Ph.D. student in UCI). I'll have posting about it with more instructions in one of the next postings.

In this posting I would like to go back to the nodes in EPN and discuss why do we need multiple nodes and what are these nodes. One version is that there is a single type of node called EPA -- event processing agent, but my preference is to have multiple nodes, since they have different roles. Here are the types of nodes in our EPN model:

• Event Processing Agent (EPA): a node that gets one or more events as an input and produces one or more event as an output.
• Event producer: An entry point to the event processing system that emits events to the EPN.
• Event consumer: An exit point from the event processing system that receives events from the event processing system
• Event channel: A node that make routing decisions within the EPN, I have recently discussed event channels in this Blog.
• Global states: This is a node that holds various type of state (reference data, global variables)
• EPN node: An EPA can represent an EPN, and thus make the EPN recursive.
  

One of the frequent questions is whether an event producer which is also an event consumer is considered an EPA. In my opinion, the answer is no. Event producers and consumers operate on the borderline of event processing network, and is actually have have two nodes one for each of its roles. The information that a consumer is also a producer is important for traceability and lineage, but we'll discuss this aspect another time.

Christoph Emmesberger, whom I met in Trento, has proposed to extend the notion of channel to provide aggregation and filtering, this idea is rooted in pub/sub systems where event processing seems as an extension of some event processing functionality. However, when we deal with event processing system, I prefer to make a clear distinction between event processing agents that is doing -- filtering, aggregation, transformation and event pattern detection , and leave channel as a routing mechanism. thus we'll ensure proper separation of concerns, and have a cleaner model. We are now finishing the chapter about pattern detection, and I'll write about patterns soon.

On event channels

Last week, the Disney Channel arrived to the Israeli cable system and enriched the set of already existing children channels; so speaking about channels, it is a good time to discuss another type of channel -- an event channel, which is discussed in chapter 6 of the EPIA book draft. Some people view channel as an edge in the event processing graph, but we view channel as a type of node, since it has some processing associated with it. We define a channel as a processing element that receives events from one or more source processing elements (We refer to EPA, producer and consumer as processing elements), makes routing decisions, and sends the input events unchanged to one or more target processing elements in accordance with these routing decisions. Note that like the term Event Processing Agent, channels are abstractions and can be implemented in various ways (e.g. through messaging systems, through buffers, through persistent stores etc...). Channels are classified according to their routing schemes. Some of the common routing schemes are:

• Fixed routing scheme: The channel has predefined input terminals wired to predefined processing elements, and predefined output terminals wired to predefined processing elements. Every event that is received on any input terminals is sent to all output terminals. Note that this type of channels can be defined implicitly.
  

• Subscription-based: EPAs or consumers can subscribe to the channel dynamically. The routing decision is determined according to the list of subscribers that is valid at the time that a decision is made.
  

• Itinerary-based: The sink's input terminal identifier or identifiers are obtained from some attribute in the event's payload, this is used to send an event to a specific consumer instance, when the EPN node is the consumer class.
  

• Context-based: The channel makes routing decisions based on the context to which the EPA belongs. This is applicable for pattern detection ("complex event processing") type of EPA. The channel selects the appropriate run-time EPA based on the context defined in the pattern- I'll discuss contexts in length in one of the next postings, as this is the topic of the next chapter in the book.
  

• Type-based: The channel makes routing decisions based on the event type of the event that is being routed.

• Content-based: The routing decision is based on the event's content, this can be phrased as assertions, rules, decision trees or decision tables, and are based on the input event content, as well as context information.

This is just the basic definition, in one of the next postings I'll show example of how all these concepts fit together.

On Event Processing Building Blocks

Back to work for one day in the office, with five conference calls (one with Germany, one with France, one with UK, and two with USA...) and then back to home for the weekend. When I have free time I like to read books, the current book I am reading is "A Lion Among Men", 0ne of the books of Gregory Macguire, who writes stories that take as background famous children stories (in this case - the Wizard of Oz), actually this is the third one behind the scene of the Wizard, now taking the Lion as its main character. I have another book of the same author still waiting...

We also submitted the draft of chapter 3 of the "Event Processing in Action" book to the publisher, which hopefully be posted on the MEAP site soon.

The approach we have taken in the book, as I have written before, is to use the "building block" approach, describing event processing principles, and the use case whose construction demonstrates the application, using building blocks, which are like the chemical elements. The application itself is being built by using "definition elements" which are like atoms (my partner for writing this book, Peter Niblett, has come with the analogy from the world of chemistry). we believe that this is the right approach to teach what event processing is -- in the "deep dive" part of the book we dedicate a chapter for each of the major seven building blocks and then dive deeper into the types of event processing agents (which deserves a different discussion). We'll also provide samples of how each building blocks is realized in different models.

The seven building blocks are:

• Event type: defines the event schema
• Even producer: the projection of the event producer over the event processing network (note that the event producer itself is outside the scope)
• Event consumer: same -- the projection of the event consumer over the EPN.
• Event channel: the glue that holds the EPN together
• Event processing agent: the brain that does the entire work; each agent is doing a specific task of processing.
• Context: the semantic partition of events and agents
• Event derivation: A building block that is possibly part of each EPA that specifies the derived event.

There are some more building blocks that are used to support these ones, but our claim is that this set of building block is what needed to build an event processing application.

Chapter 4 which is in advanced phases of being written starts the deep dive by discussing the event type building block, and in one of the next posts I'll say more about it.