On Innovation, Twitter and Event Processing

Reading in the newspapers about some troubles in Thailand reminded me about my family trip to Thailand two years ago, country that is full of paradoxes, but one of the most attractive to tourists, tourism is a major industry in Thailand and the local people have very innovative ideas for tourist attractions, one of them is to let people touch tigers, an animal known as one which is not easily tamed, here is a proof.

Talking about innovation, two recent postings recently motioned Twitter in conjunction with event processing, the one by Richard Veryard talked about "innovation in a small bakery" describing a small bakery which sent Twitter notification whenever freshly baked bread came out of the oven, and wondered why we don't see more innovation like this from IT departments in organizations. The second is a discussion thread that started by David Luckham on his CEP forum asking whether CEP can be used to trace Twitter notes.

Some people have already answered David, I guess that one of the difficulties is by the fact that Twitter messages are written in a free text, however, trying to apply full scale information retrieval techniques with statistical reasoning will miss the point of simplicity, the main consumers of such applications are individual consumers that cannot apply complex software and reasoning, one cannot kill flies with a cannonball. So what can be done: the messages (like in the bakery example) should be fixed and include keywords that can be easily filtered. For example, the bakery may have three messages about three products: bread, rolls and cakes and can also send alert when each of them have been fully sold. The bakery can provide a way to subscribe to each or make more complex subscriptions, and here comes event processing patterns to be used for example: I want to go to the bakery only when there is both fresh bread and fresh roles are available (and not sold out yet), which is a conjunction pattern, or I want to restrict it if the conjunction of the two event happened within 15 minutes, since I like it very fresh... In this case the identification of the event type can be done by a simple filter that search for the keyword "bread" or "cake" together with the event source (bakery) and time-stamp. If more stores join this type of service, I can do a conjunction of Twitter messages from both stores buying fresh bread and fresh shrimps in the same trip (adding some condition on the distance between the stores, given by an additional service. Furthermore, I can also condition any subscription in my personal context (e.g. only when I am at home, only in the morning hours etc...), and this is initial ideas, so for sure, Twitter as a platform for event notifications can have many usages (which does not say that every Twitter message carries event information).

Richard's question about innovation and IT department is more complicated one, however, there is some truth in his observation, based on my own experience in an IT department of a large organization, IT departments may be more conservative than their users, and typically will be cautious about be anything that is giving a "programming capabilities" to the users (and is conceived as "losing the control"). Since many event processing applications' value to the business users is doing exactly this (giving more power to the business user to do their own programming) it is sometimes a challenge to get it through the IT department, but this is a longer discussion...