Start small, succeed and then scale - Mark Palmer's best practice on employing event processing in an enterprise

One of the observations is that it is not trivial for enterprises  that are used to legacy systems and legacy systems and legacy developers to adapt to event processing thinking.  I have written about it recently.  A recent meeting with a customer triggered the initial reaction - "but we don't know how to work this way", the way they know how to work is: getting events, inserting them to a database, and ask periodic and on request queries.   I have also recently compared the request driven (legacy) approach and the event driven approach. 

Mark Palmer, CEO of Streambase, has recently written a "best practice"  on this topic of how to introduce event processing in an enterprise, and entitled the best practice "Start small, succeed and then scale".  

Mark starts by asserting that real-time becomes a must do agenda in many enterprises, and many type of applications, but the problem is that both legacy systems and people require an overhaul. 

His methodology is - choosing a small, but important application, put the best people on it, iterate if failure, learn from experience, and then scale it to other functions and applications.    I think that this is a good approach, it would also be interesting to have some experience reports about application of this best practice in reality.     I'll write more about this topic - later.

Dagstuhl seminar on event processing - the second day

This is the Dagatuhl logo, we are about to start the third day -- a shorter day in terms of work, since we are doing excursion for half a day today. In the second day: two sessions of breakouts in groups, I am visiting each session a different group, and two "deep dive" session -- the first one on "why" event processing, in which they have tried to come up with cost/benefit model, and the second is the event processing grand challenge, in which they presented "smart society" as a main theme, with variety of "smart" application, but the main message is that while each application in the list may be a challenge, the grand challenge is the holistic view, namely, the interdependencies among all these systems.

In the evening, over some wine, beer and cheese, there was a more informal session (no slides) in which different vendor representatives told us some lessons:

Marco Sierio from ruleCore started in eight lessons he devised in the train on his way, one of them is that people have hard time to adjust to the "event driven" thinking, since they are programmed to think in "request/response", Marco will probably put all eight lessons on his Blog, but one of the interesting lessons is that reading research papers is a time well spent, although it is not an easy task to do.

Richard Tibbetts from Streambase also provided some insights from his experience, ending with a statement that he did not see a lot of demand for pattern detection with his customers. Maybe in trading applications patterns detection is not a natural thing to do ?

Badrish Chandramouli from Microsoft provided some insights about StreamInsights. The interesting feature is the temporal model, which allow events to be defined as point event, interval event, and interval events with fuzzy boundaries. They also allow speculative computation and out-of-order processing, at least to some extent.

Alex Kozlenkov from Betfair, is both an user, and a developer of Prova, a self-made event processing platform, that follows the EPN/EPA model that we describe in the EPIA book (well - some variation of it) and explained its features in his usual enthusiastic way.

Martin Hirzel from IBM System S, talked about some of his impressions in working on that team, Martin is a programming languages person, working on the SPL (Stream Processing Language) and provided some insights about building a language by multi-disciplinary team.

Udo Pletat also from IBM, but from the software services organization, provided some insights on applying RFID oriented applications with customers (this is based on the Websphere Sensors Events product of IBM which has embedded Amit), and provided some examples of why they needed to use event processing patterns to track people's movement in chemical plants with restricted zones.

The session ended after 11pm and was very interesting gathering. Today -- the deep dive about "what is event processing" - more later.