Saturday, April 18, 2009

On Event Processing Building Blocks

Back to work for one day in the office, with five conference calls (one with Germany, one with France, one with UK, and two with USA...) and then back to home for the weekend. When I have free time I like to read books, the current book I am reading is "A Lion Among Men", 0ne of the books of Gregory Macguire, who writes stories that take as background famous children stories (in this case - the Wizard of Oz), actually this is the third one behind the scene of the Wizard, now taking the Lion as its main character. I have another book of the same author still waiting...

We also submitted the draft of chapter 3 of the "Event Processing in Action" book to the publisher, which hopefully be posted on the MEAP site soon.

The approach we have taken in the book, as I have written before, is to use the "building block" approach, describing event processing principles, and the use case whose construction demonstrates the application, using building blocks, which are like the chemical elements. The application itself is being built by using "definition elements" which are like atoms (my partner for writing this book, Peter Niblett, has come with the analogy from the world of chemistry). we believe that this is the right approach to teach what event processing is -- in the "deep dive" part of the book we dedicate a chapter for each of the major seven building blocks and then dive deeper into the types of event processing agents (which deserves a different discussion). We'll also provide samples of how each building blocks is realized in different models.

The seven building blocks are:
  • Event type: defines the event schema
  • Even producer: the projection of the event producer over the event processing network (note that the event producer itself is outside the scope)
  • Event consumer: same -- the projection of the event consumer over the EPN.
  • Event channel: the glue that holds the EPN together
  • Event processing agent: the brain that does the entire work; each agent is doing a specific task of processing.
  • Context: the semantic partition of events and agents
  • Event derivation: A building block that is possibly part of each EPA that specifies the derived event.

There are some more building blocks that are used to support these ones, but our claim is that this set of building block is what needed to build an event processing application.

Chapter 4 which is in advanced phases of being written starts the deep dive by discussing the event type building block, and in one of the next posts I'll say more about it.

Wednesday, April 15, 2009

More on Decison Management


Today is the last day of the 8 days holiday, and tomorrow - back to the office (well for one day only, since our weekend in Israel is Friday and Saturday, and Sunday is a regular working day), and I have a huge "to do" list that keeps accumulating, and a bunch of phone calls... Today, however, I have travelled with my family to Bet She'an, one of the biggest archaeological sites in Israel (around 70 KM from Haifa). The roman city was exposed and much of it has survived, as you can see in the picture above (there is much more).

Today - just a short posting as a follow up on James Taylor's posting, who is advocating for a while the notion of decision management. The following slide is copied from JT's posting and explains the difference between decision management and decision support




The difference, as shown, is between helping people to make manual decisions, which is the BI land, and make automated decisions, which is the DM land. One of the topics I am working on recently is the notion of "decision" as an abstraction which should be programmed directly by business users, and will hide a blend of technologies such as: BRMS, event processing and various kind of analytics; I believe that the next wave of computing will be centered around the ability to business users (or direct consumers) to program and the role of IT will be replaced by autonomic computing techniques -- we are not there, but this is something to expire. Event processing will play multiple roles in this setting, but I'll leave this discussion for another time -- it is late, and I want to advance in the book I am reading since tomorrow will be busy..

Tuesday, April 14, 2009

On Innovation, Twitter and Event Processing


Reading in the newspapers about some troubles in Thailand reminded me about my family trip to Thailand two years ago, country that is full of paradoxes, but one of the most attractive to tourists, tourism is a major industry in Thailand and the local people have very innovative ideas for tourist attractions, one of them is to let people touch tigers, an animal known as one which is not easily tamed, here is a proof.

Talking about innovation, two recent postings recently motioned Twitter in conjunction with event processing, the one by Richard Veryard talked about "innovation in a small bakery" describing a small bakery which sent Twitter notification whenever freshly baked bread came out of the oven, and wondered why we don't see more innovation like this from IT departments in organizations. The second is a discussion thread that started by David Luckham on his CEP forum asking whether CEP can be used to trace Twitter notes.

Some people have already answered David, I guess that one of the difficulties is by the fact that Twitter messages are written in a free text, however, trying to apply full scale information retrieval techniques with statistical reasoning will miss the point of simplicity, the main consumers of such applications are individual consumers that cannot apply complex software and reasoning, one cannot kill flies with a cannonball. So what can be done: the messages (like in the bakery example) should be fixed and include keywords that can be easily filtered. For example, the bakery may have three messages about three products: bread, rolls and cakes and can also send alert when each of them have been fully sold. The bakery can provide a way to subscribe to each or make more complex subscriptions, and here comes event processing patterns to be used for example: I want to go to the bakery only when there is both fresh bread and fresh roles are available (and not sold out yet), which is a conjunction pattern, or I want to restrict it if the conjunction of the two event happened within 15 minutes, since I like it very fresh... In this case the identification of the event type can be done by a simple filter that search for the keyword "bread" or "cake" together with the event source (bakery) and time-stamp. If more stores join this type of service, I can do a conjunction of Twitter messages from both stores buying fresh bread and fresh shrimps in the same trip (adding some condition on the distance between the stores, given by an additional service. Furthermore, I can also condition any subscription in my personal context (e.g. only when I am at home, only in the morning hours etc...), and this is initial ideas, so for sure, Twitter as a platform for event notifications can have many usages (which does not say that every Twitter message carries event information).

Richard's question about innovation and IT department is more complicated one, however, there is some truth in his observation, based on my own experience in an IT department of a large organization, IT departments may be more conservative than their users, and typically will be cautious about be anything that is giving a "programming capabilities" to the users (and is conceived as "losing the control"). Since many event processing applications' value to the business users is doing exactly this (giving more power to the business user to do their own programming) it is sometimes a challenge to get it through the IT department, but this is a longer discussion...