Revisiting the FFD example in the EPIA book - call for contributions

In 2009 when Peter Niblett and myself wrote the "Event Processing in Action" book we came with an example called "Fast Flower Delivery" that accompanied the book.   it seems that the book is still being used (it was cited so far 276 times),  one idea that came recently in discussions with the publisher is to refresh the implementation page.   We now issue a call for implementations.   All owners of event processing tool (commercial or open source) is called to show how this example is developed in their tool so the reader (and interested general audience) will be able to view it.   In order to qualify one has to create a webpage that contains - source code of the use case with any additional reference to the language and product behind it.  

The book's webpage will include links of all such solutions.  For more details -- please contact me.  

The full details are the FFD example are linked here. 

On airport events

Tomorrow I am planned to visit three airports,  starting from Ben-Gurion airport near Tel-Aviv to takeoff from Israel, moving through Frankfurt airport in Germany for connection (hopefully no serious delays, it seems to be snowing in Frankfurt),  and ending in the Dulles airport near Washington DC.   I have reported in this Blog in the past on several flights related events, most recently - getting back from DEBS 2012 in Berlin 49 hours later than planned.   Airports are full of events,  and it was interesting to read Pedro Garcia blogging about  event processing as part of airport management system.    In an airport there are many events related to flights, luggage, strikes, weather and more.    The events that Pedro relates are both event that happened and predicted events (flight delay can be predicted).    While some of the delays are force major, others are certainly created by people (I have plenty of stories about human inflicted events).   Improvement of flight management, as well as improvement of luggage handling systems, can be very helpful.

It is also interesting to note that the system is based on the architecture we presented in the EPIA book, which is explicitly mentioned.  Always good to see that our work is useful to somebody :-). 

Hoping for uneventful flights (have several more flights during the coming week). 

On IBM scientific accomplishment

This week, the annual accomplishment process of the IBM Research Division was concluded. This is a process that recognizes major impact activities in various categories: scientific, contribution to IBM products, contribution to IBM services, contribution to standards and some more.  

Within this year's process, our work on the event processing conceptual model has been recognized as a scientific accomplishments.  The criteria are: number of citations (according to Google Scholar) and support letter from senior members of the scientific community in this specific area. 

It is interesting to note that the major publication referred was the book I have written together with Peter Niblett, "Event Processing in Action".

 The interesting fact is that the book was not written as a research oriented book, but was geared towards the professional market,  yet it accumulated so far 153 citations, with the number steadily growing (when the process started the number was around 130).  

Drilling down to the citations list it is also interesting to observe that while some of the citing papers belong to the event processing community, many others come from different domains and implemented systems in the areas of power management in mobile devices from Finland, rotor-craft control from Brazil, as well as others that indicate that the material in the book had some practical impact in additional to the impact on the scientific community, which is also important, as science is being built in layers.

I have been out of the research work for about 10 years, where I kept research activity in the back sit, mainly through supervising  PhD and MSc students at the Technion.  The major project I was involved in the years 1998-2005 (AMIT), has a single major publication that is actually the summary of the PhD dissertation of Asaf Adi  (this paper also accumulated nice number of citations). 

The question whether citation number is a good metric - is another discussion, for me the actual impact (those using the work in practice) is also an encourging indication that the work is not done in vain -- more later

StreamEPS from SGT - an open source event processing from Ghana

I was recently approached by a company that resides in Accra, Ghana called SoftGene Technologies,  which  has developed an open source event processing product called StreamEPS.   Looking closely at the description of the supported functionality, one can realize that this is an implementation that follows the EPIA book.  I'll write in a separate posting about the impact of the book in terms of follow-up works, it is quite interesting...

Softgene technologies describe itself as "Research-lead private company".  I like the definition, since I believe that much of the useful software is research lead. 

This also completes the continent coverage of people working in development event processing software.  

While there are quite a lot of software developed in Europe and North America.  There is now event processing software developed in Asia (Sri Lanka, Japan and Israel - that I know of), Australia, and Brazil.

If there are event processing related software developed in additional countries -- let me know and I'll survey in this Blog.

On context and punctuation

This illustration is taken from the EPIA book and shows the notion of context as a first class construct that groups together events that need to be processed together, where each of its instances is processed as a single unit,  context may have temporal aspects, segmentation aspects, spatial aspects, and state-oriented aspects, and the actual context instance may be combined of one or more aspects.  

Recently I had some discussion with somebody from the data stream community who was in the opinion that the stream oriented way to do it is more simple since they are based on standard thinking (i.e. SQL).

In the stream research literature, the temporal context is divided between windows and punctuation, all the other aspects are somehow expressed by the group-by construct.    I guess that windows and group-by constructs are familiar to many people, punctuation is less familiar, thus I'll explain the idea briefly. 

A good source to learn about punctuation is Peter Tucker's site, from where the illustration below is copied:

Peter Tucker has done his PhD (under the supervision of Dave Maier) on data stream punctuation;  the idea of punctuation is to enable creating sub-streams, since a stream by definition is infinite.   The notion of windows in classical data stream models consists of variations of sliding windows, either by event count, or time, however, in reality there is a need to explicitly end stream, i.e. when the bid ends.  The definition of 
punctuation in the database encyclopedia (the value was written by Maier and Tucker) says:

In other words,  this is a "dummy event" put into the stream to denote the end of a sub-stream.  


Now the question is whether using windows + punctuation + group-by is indeed simple.    The relational model's claim to fame is its simplicity, however, from early days of the relational model, it has been realized that there are semantic anomalies, and all the normalization rules came to resolve those anomalies, yet, the overall model has been simple.  However, this is not really true for the extension of the relational model for streams,  while the relational model has a single entity: relation,  the extension has multiple entities: relation, stream, window.  Punctuation is a kind of trying to add semantics as a kind of logical patch.  


IMHO, the context model is semantically cleaner.  It has a single semantic notion for all the constructs that determine how to group event together (in the stream terminology, create sub-streams).   In the context model  there are no dummy event; the notion of temporal context (window) can also support event intervals - time windows that start by event and end by event; the events that start and end the window are real events, and can be used for other purposes.  Moreover, looking at the definition of punctuation above, it defines the end of a sub-stream, but assumes that the next bid starts when a new bid event arrives.  However, if we look at a situation where bids have not only "end bid" events, but also "start bid" events, then we would like to ignore bids that arrive when no bid is open,  and this has no natural representation, and has to be implemented with tricks, so we also need the equivalent of  "start sub-stream punctuation".   So, we view this as kind of context based on , and not as kind of event within the stream with a special semantics. 


The model we presented in EPIA (which we keep evolve)  to be a semantic model that provides abstractions above the current implementations of both event and stream processing systems;  several product owners have already told us that they use concepts from the book as inspiration to next versions of their products, and I guess that full implementation that is based on this model as native is yet to come.   

Another implementation of the "Fast Flower Delivery"

In the EPIA book, we had a running example used for demonstrating all constructs in the book, the example described a scenario called:  "Fast Flower Delivery".    During the book writing we approached the event processing community and issued call for implementations, there has been six implementations that were ready  during the book's writing:  Aleri (currently Sybase),  Apama(Progress),  Esper, Etalis,   ruleCore and Streambase.    It seems that more implementations are being devised,    I was asked for permission to use the "Fast Flower Delivery" scenario as the running example in an upcoming book teaching the use of one of the products,  will write about that when this book will be out. 


Recently,  an implementation of this scenario in IBM Websphere Business Events (WBE) was posted on IBM developerWorks as a tutorial to teach the use of that product.  
Seems to becoming the "Hello World" of event processing.  

A course based on the EPIA book

I got a request today to share the slides that I am using for the event processing course at the Technion,  based on the EPIA book, in order to use it for an event processing academic course.    At this point I will share it in a controlled way for those who wish to use it to give a course,  if you are interested to give a course based on this book, please let me know and I'll give you access to the course's site. 

On the boundaries of windows

Today I heard rain knocking on the window.  Rain in June is quite a rare event in Haifa, but happened before.  When I lived in the USA it was always peculiar to me to watch heavy rains and thunderstorms in the summer. 


Looking at windows,  window in event processing is an important concept, designating temporal context.  we have discussed this issue in length in chapter  7 of the EPIA book,    Windows can be isolated or sliding, isolated windows can start by event or fixed time, end by event or fixed time, or expire after some time offset.   
Sliding windows can slide by time, or by event count, and can be overlapping or non-overlapping.  


In any of these variations, every single window is a time interval.    The question is what type of interval - open or closed on both ends.    


In the book we mention two types of intervals:


For most types of window, we used the half open window,   if the interval boundaries are denoted by Ts and Te (for start and end),  then an event whose time-stamp is  T belongs to the window if  Ts ≤ T < Te, which says that events that occur in the interval starting point are included, and those at the interval ending point are not included.   This, for example, guarantees, that in non-overlapping sliding time window, an event belongs to exactly one window instance.   
For the sliding event window we used the close interval semantics Ts ≤  T ≤ Te.  The rationale is that if the sliding window has a count of five events, we typically mean that all of those five events belong to this window. 


Some comments here:  I have heard the opinion that it is not an issue, since there are systems which create total order of events, serializing them, by having a single process that assigns time-stamps.     These can be valid for some applications, but is not valid in the general case due to two reasons:

1. The event that starts or ends the window can by itself participate in some EPAs that are active in this window, thus a decision is needed whether it participates or not.
2. In various applications the applicable time-stamp is the occurrence time of this event as reported by its source, and not the detection time assigned by the system, thus several events can occur at the same time-point.  Furthermore, even in the assigned detection time,  in distributed systems there may be multiple entry points, and ensuring total order may not be cost effective.   

While in the book we mentioned two of the four possibilities for time intervals, one can think of cases in which the other two may be useful, which indicates that such semantics might be required to be configurable. 

Some catching up

Have not written for the last week,  spent some of it in the IBM La Gaude site, near Nice, in South France, the La Gaude site is on mountain and can be seen in a bird eye's view here.    The meeting was an internal meeting of "technical leaders" (whatever it is) in IBM South and West Europe, and was kind of educational and networking session.    Back home - occupied by some family matters, so now I found some time to catch up.    First looking at the Blogland,  one can find two old bloggers returning after long silence, one of them is Brian Connell,  who returned to blogging after a long time, the other is Marco from Rulecore, who also returned to blogging after a long silence.  In a comment to my Blog,  the Rulecore guys are wondering why Rulecore is not mentioned in the ACM Computing Survey upcoming survey on event processing while other are mentioned.   I think that the reason is quite obvious, while I am familiar with Rulecore (they participated in the implementation of the EPIA example; my students used them in project several times),  if one wants to be known by the academic community, then one has to participate in the research oriented conferences, publish (at least industrial) papers etc --- in our world, visibility is the name of the game, so you might consider sending an industrial paper or experience report to DEBS 2011,  or participate in the DEBS 2011 challenge.    


Another blog (Jason Irwin's)  that I don't typically follow provides a new review of the EPIA book.     


My next trip is planned for late March in the USA,  I was invited to give a tutorial on event processing as one of the highlights of the OMG Technical Conference,   I might also do some other meetings in the USA (customers' meeting, IBM meetings, university visits),  but still working on the plan.     

My 2010

In 2010 I have posted less entries in this blog, relative to 2008 and 2009, 128 postings, roughly once every three days. Maybe I've been busier, and maybe I've been lazier - probably the combination of these two. 

The happiest day of the year was surprisingly the day in August when I've returned from a family vacation in western Canada and found two things that got by mail:   

12 copies of the book EPIA that fresh out of press (I have already gave 10 of them as a present, left 2 for myself - one at home and one in the office); the EPIA book was a major task for 18 months, so I was relieved to see it in print, at some points it fell like never-ending saga. 

 

The second item that arrived it the same day is a plaque designating the fact that I've received an IBM corporate award, which is the highest award that IBM gives,  the order of the events was somewhat funny: first I got the plaque in the mail, two weeks later I got a letter signed by IBM's CEO, which notifying me that I am receiving the award, two weeks later than the list of awards was publicly published, and then after six more weeks there was the award granting ceremony, giving me again the plaque I got in August.

Another notable event was the Dagstuhl seminar on event processing in May 2010.   We are now in the final phase of editing the end result of this team work.  Dagstuhl is a wonderful place, and we had a very good team there spending  5 days in dealing with the present and future of event processing.

I had some trips abroad - both business and pleasure.   Unlike most years I spent only four days in the USA in October, for the OMG financial market conference,  spent two and half weeks in a family vacation in Western Canada,   here is a picture with my daughter Daphna somewhere in the Canadian Rockies.

Another trip was to VLDB in Singapore, with a couple of days vacation in Hong-Kong on the way, and several days vacation in Singapore.

Here is a picture from Hong-Kong's wax museum,  where I am photographed with an old friend.

Here are two pictures from Singapore, one from a zoo that resides in a rain-forest in the northern part of the island, and the second in a spa where you can get your feet cleaned by fish

Another conference I have participated in was DEBS 2010 in Cambridge, UK.

 This year I have also started the work on a new project that deals with proactive computing, and will write more about in in 2011.    

Overall --  interesting year,  the leaves a lot of unfinished challenges for the future.  

On ACM Distinguished Speaker Program

Today the ACM Distinguished Speaker program announced my inclusion in the list of "ACM Distinguished Speakers".   The program is described in the ACM DSP site,  while this is a big honor, especially looking at list of speakers that include some real giants,  it is not a recognition program, but a program that has a mission stated as:  The DSP is an outreach program if ACM that brings distinguished speakers from academia, industry and government to give presentations to ACM chapters, members and the greater IT community. 


The outreach mission means that by accepting the nomination to ACM Distinguished Speaker, I commit to 
travel and give talks by request of local ACM chapters worldwide, there are some ground rules that can be found on the site,  e.g. to justify international travel (fully funded by ACM) there should be an accumulated audience of 300 people, so it typically entails multiple talks during a single trip.     I am in the opinion that I should spend some of my time in sharing knowledge with the greater community, this is the reason I am teaching, providing long tutorials in various conferences, and wrote (together with Peter Niblett) the book "Event Processing in Action".    Thus, accepting this nomination is another link in the chain, and I'll try to do my best to satisfy requests, especially from places in the world which don't get a lot of talks on the event processing area.


My speaker page shows four proposed talks, three of them deal in event processing:

•  A short tutorial that serves as introduction to event processing -- summarizing the material in the EPIA book.
• A talk about the research challenges that exist and a "call for action" to the research community in a way to move the event processing area towards its next generations
• A talk about proactive computing, one of the extension directions of event processing, on which I concentrate recently.

The fourth talk is on more general theme:   Computer Science Research in Industry -- some history and different models of how it operates.


I hope that it will be both useful to the audience and fun. 

Teaching an event processing course again

Yesterday I have started teaching again, this time in the Bloomfield building at the Technion, seen in this picture.   Sometimes people ask me why do I continue teaching,  it is probably not because I am bored, on the contrary,  I am quite busy;  it is probably not because it is a well-paid job,  actually they pay peanuts.  
I think that I continue to do it, since I like the interaction with students, and it is also means to evangelize "event processing" to the younger generation;  it also provides exposure to graduate students to work with, as over the years I have supervised (or co-supervised)  6 PhD dissertations and 21 MSc thesis, if my count is right.   It is also source for recruiting.   Quite a lot of my colleagues in IBM have been my own students (which I supervised), or my teaching assistants, or students that took courses with me.    


The way my courses are designed revolve among the following principles:

For historical reasons the course is called "database engineering" and always there is a student who expects to see classical database course,  for these students I answer that today in any database conference, data stream management is one of the emerging areas, and I am giving a flavor of it (my approach is not database centric,thought), I also show students some analysts testimonies that event processing is an emerging markets in general.   The second principle is deep dive -- the course is going more on depth then on breadth, the students should gain deep understanding in what event processing is. The last principle is retain over time, my own experience is that to retain over time students have to experience themselves, so they'll get implementation project, I have not yet decided what the projects will be, gave myself time until next week.
I am using the EPIA book as a text book,  but this time will not focus on the FFD example.   
I might create teaching materials for this course so that others will be able to reuse it.


  

New review for the EPIA book

It is holiday again, and tomorrow I am leaving for a short (2 days) vacation here in Israel with my family, and then in Sunday night, travelling to the USA for a short (4 days) business trip, where the highlight will be participation in the event processing - capital markets conference of OMG EP CoP;  here is the conference's program. I have been travelling too much recently, hope that after this trip I'll have a break in travelling.


Today I've noticed a new review of the EPIA book posted by Tushar Jain, a person I have not been familiar with so far.   Good to see that people like the book.   The reviewer is right that the conceptual model we described in the book still need to obtain acceptance like UML and BPMN,  well -- the next step is to try and work on standard proposal for event processing modeling language, and I'll take advantage of my coming trip to USA to try and kickoff this activity.
One comment -- I don't see the other books mentioned as competitive.  The book of Mani Chandy and Roy Schulte is a business oriented book, and our book is a technical oriented book, so the intersection is fairly limited, furthermore, we mention in the book that we don't deal thoroughly with the business perspective, and recommend Chandy and Schulte's book as a complimentary for those who would like to get deeper understanding of the business perspective.      Later in October I'll start teaching again a course in the Technion based on the EPIA book (first one since the book is out, though I have used the book's draft for previous course).

The book: Event Processing IN ACTION - is now out

I still need to write something about the vacation in Western Canada, but returning to the office today, I have received a package of copies of the EPIA book, that was just published. The project of writing this book (in my spare time) was quite demanding, and was twice longer than the original expectation. I have talked with some colleagues who wrote books for other publishers recently, and found out that relative to their experience, Manning has exceptional quality control procedures, with three reviews by readers during the book's development, and a multi-stage production process with a lot of iteration between the authors and various people on the production team -- technical proofreader, copy editor, proofreader and the production manager.

The acknowledgements section of the book also lists many people who helped in contributing ideas and review and the Manning team; I am also grateful to David Luckham who agreed to write the Forewords section. Last but not least -- working with a partner on such a project requires the ability to agree on many details, and a lot of interaction, many of them in evenings and weekends. My partner in writing this book, Peter Niblett, has complemented me since he came from a different perspective; Peter's drive for perfection has contributed considerably to the quality of the book, Peter is also a very pleasant person to work with.

I also noticed that Manning added a section explaining who is the person on the front cover (some people asked me).

Manning maintains an Authors forum that enables communicating with the authors, this forum helped us during the book's development process to get feedback, and a lot of the comments have been adopted (with acknowledgement to the appropriate person); this forum is being kept alive.

The book itself is printed in black and white, there is also a colored ebook version, available on the Manning's book webpage. The book also included a use-case that is known as FFD (Fast Flower Delivery), and has already several implementations in different languages, with more expected, this site will be kept as live site and is being hosted by EPTS. Interestingly, the FFD case has been used within the DEBS 2010 event processing architectures tutorial in order to demonstrate the architecture notions.

The book has served as basis for both academic and IBM internal courses, and I'll be able to share teaching material with anybody interested to use the book for that capacity.

Some other follow-up ideas is to provide comprehensive authoring tool for the model described in the book, work on automatic compilation from the book's model into various event processing languages and maybe to general programming languages as well -- these can all be nice students' projects, the model described in this book can also serve as a first iteration on standard in event processing application modeling. We'll see how much of the follow-up will be materialized.

On the production phase of the EPIA book

I have co-edited some "collection of articles" books that are common in the research community, there is certainly some logistics associated with it, but relative to producing a "mass market" book it is a children's game comparing with the production of the EPIA book that I have been writing together with Peter Niblett. The publisher of our EPIA - Manning, has various ways to ensure quality. Manning is a publisher focusing on computing related book, thus it views book development as a software development project (and indeed many of its books are code-driven). While the book was initiated by Manning and not by us, they have sent the outline we sent to bunch of reviewers, all of them people with deep knowledge in event processing. Some of the advices we got were very useful. Since we wrote in the outline that we are going to base the book around a single example, the advice of one of the reviewers was that we'll use an example that everybody can understand, the reviewer added that if we would chose an example from the financial services domain (as many of the EP papers are doing), we might create a communication obstacle with some of the readers, who will not be familiar with the terms. We followed this advise and created the "Fast Flower Delivery" (FFD) that already received multiple implementations in many languages. For each 1/3 of the book, we had a milestone in which they sent the book to a bunch of reviewers (in one of them there was a response from 14 reviewers). The reviewers were mix - some people who have deep knowledge in event processing, and some who don't know what it is, both types of reviewers sent various types of comments, some of them resulted in restructuring of the book (originally we planned 15 chapters and 3 appendices, we ended up with 12 chapters and 2 appendices, but with around 100 pages more than we originally planned)- This was just the development process, and it last for 15 months. A few weeks ago we have started the production phase -- it has a workflow (supported by electronic content management system): Another technical review by a "technical proofreader" - here we looked at PhD student in this area, and thus addressed faculty members active in this area; the person who did this job is Samujjwal Bhandari, a PhD student in Texas tech university, he has read all chapters, made comments and caught various cases of inconsistencies among the different chapters, then it moves to the copy editor who makes editorial modifications, and returned to us with many comments and questions that we had to answer, after doing this round, each chapter is going to the proofreader, who is doing another editorial pass, and then comes back to us with questions and comments, so we are doing another pass on all chapters. After that it goes to the typesetting. In the background there is a graphical work to redo our amateurish figures in a professional way, and then hopefully the book will be ready. Currently we passed the phases of the copy-editing, and now working with the proofreader. There are a lot of editorial rules, and style issues that have to be dealt with (Peter is much better than me in noticing the small details), and in fact -- amount of work is much higher than I anticipated, it consumes much of my free time for over a year now, as we both are doing it in addition to our daily work -- but I hope the result will justify the investment. More about the book - later.

Some media events

I have written last week about participating in an entrepreneurship panel, where there has been a discussion on whether innovation in big companies is counted or not, it turns out that the echo of this discussion got to the national press in Israel, although you have to read Hebrew in order to understand what is written. Well - for those who do, they did not quote me accurately, and even did not spell my name correctly... go figure.

And from past media event to future one, in Thursday I am participating in an IBM Webinar given on the ebizQ platform. The Webinar is a blend of the IBM "business agility now" message and its relations to event processing, and some selected topics from the EPIA book. The Webinar is planned for Thursday, March 25th at noon USA EDT. You are all invited.

On spatial relations between events and entities

During the final pass of revisions for the EPIA book, we have re-visited the issue of spatial context, one of the options of spatial context is to group or partition events according to the relationship between event and a certain entity. However, location is not necessarily a point, there are three main type of location: point, line (or poly-line) and area (polygon). This is true for entity, but it is also true for an event, if the event is located using GPS, then a point coordinate is obtained, but if an event is located using the signal sent from a cellular phone, then the location of the event is a cell, i.e. it occurs within an area. We can also say that a vehicle is somewhere on road M1, which is a poly-line. Given these three data types, we can define several relations among the location of the event, and the location of the entity, as shown in the illustration above, such as: contains --- the entity contains the event, disjoint etc... The illustration above shows the various cases between events of various location types and entities of various location types. Some examples of use cases for the various relations will follow in one of the next postings, meanwhile -- you can think about applications yourself

On common misconceptions about event processing - the single application misconception

We start the introduction chapter for the EPIA book, by stating: Some people say that event processing is the next big thing; some people say that event processing is old hat and there is nothing really new in it. Both groups may be right to a certain extent. As with any field that is relatively new there is some fog around it: some of the fog stems from misconceptions, some from confusing messages by vendors and analysts, and some arises because of a lack of standards, a lack of agreement on terms, and a lack of understanding about some of the basic issues.

In the book we don't really talk about the misconceptions, but I think it is a good topic towards the end of 2009 to dedicate some postings towards the major misconceptions.

I'll start with misconception number 1: Event processing is a single-industry (some even say single-application) technology, and event processing software cannot generalize beyond this single industry/application.

The industry is, of course, capital markets, and the application is algorithmic trading

The diagram below is taken from the ebizQ customers survey (two years ago) about what are the business problems that they expect to solve with event processing, and the result is 9% indicated algorithmic trading.

This misconception is originated from the fact that the capital market industry has indeed been the early adopter of event processing software, and served as a proof of concept for the rest of the industries, there are indeed some vendors that focus mainly around this type of application, however, this does not show the entire picture. From the IBM experience I know of customers in various industries, most are not in the capital market area. Getting to the material collected by the EPTS use case work group (the material is on the EPTS members internal site, available to EPTS members) I find quite a lot of examples of systems working in production or being developed from variety of domains, here are some samples:

• Border security radiation detection (Eventzero)
• Mobile asset geofence (Rulecore)
• Logistic and scheduling application (Starview)
• Unauthorized use of heavy machinery (Rulecore)
• Hospital patient and asset tracking (IBM)
• Activity monitoring for taxing and fraud detection (IBM)
• Intelligent CRM in banking (TIBCO)
• EDA and asynchronous BPM in retail (TIBCO)
• Situation awareness in energy utilities (TIBCO)
• Situation awareness in airlines (TIBCO)
• Reduce cost in injection therapy (IBM)
• Next generation navigation (CITT)
• Real-time management of hazardous materials (Oracle)
• Finding anomalies in point of sales in retail stores (CA)
• Elderly behavior monitoring (U. of Munich)

These are only samples, I am familiar with variety of examples in various industries: healthcare, utilities, chemical and petroleum, insurance, security, transportation and others. In the last event processing symposium in Trento, we had one keynote address on event processing in robotics, and there are other areas as well such as smart house to monitor energy consumption in the house. While we are now in the first generation, and the utilization of event processing will increase in time, the coverage in terms of industries and applications is growing has gone far beyond algorithmic trading.

More misconceptions in subsequent postings.

More on the ecosystem of event processing systems

The one week school break for children ends tomorrow, and today I went with my two younger daughters to see Avatar, a very ambitious movie with a lot of advanced 3D graphics. The downside is that the film spans over 3 hours without a break -- could cut 1 hour easily, but nevertheless the graphics is very impressive, the plot is kind of paraphrase on other movies.

In my previous posting I've written about event processing functions, one of the frequently asked questions is whether event processing has a value by its own right, or as part of a larger ecosystem. Let's look at another question: DBMS software deal with update, retrieve, store, organize, backup and restore data, it also supports concurrency control, transaction management and other stuff. DBMS itself does not have value, the value is in the use of data.

In a similar way, event processing software deals with transferring, filtering, transforming, pattern matching, and situation discovering, it also provides execution control and support of some non functional properties, but the aim is to receive event process them and derive more events. The value is not in derivation of events, but in the way it is used.

In order to complete the cycle we need two other types of software: event producers that produce the raw events, and event consumers that consume the derived events and execute the actions triggered by these events -- the consumers are those who are driving the value.

The producer can be any type of software or sensors. One example of event producer I've written about in the past is the "EDA extension of CICS" that I have written about before. It is an example in which a software is instrumented to produce events. We are seeing more software of this type in the sensor area, and other areas as well.

Consumers are the part of ecosystem that utilize the events flowing from the event processing systems. I've posted before about the consumer chapter in EPIA, There are many ways to consume events, they can also be connected with other types of enterprise software: Events can drive decisions, and can serve as input to business rules; events can drive KPI (Key Performance Indicators) metrics and can be input to BAM systems, it can have various actions related to BPM systems, events can drive real-time analytics and be input to BI systems, it can post on Twitter, and update Ambient Orbs (see the posting about the EPIA chapter), or if it is part of a "smart house" it can turn off lights or control the temperature using some actuators.

Like DBMS that requires somebody to feed the data, and somebody else to use it, event processing requires its ecosystem, we'll see more of the ecosystem software both in the producer side (like the CICS example) or in the consumer side (like BAM software). More on event processing ecosystem later.

On EPIA Website

The Event Processing In Action book that Peter Niblett and myself are writing is getting to the last phase before getting to production. We have finished a full draft that is now going to through the publisher's review system. The book has a closely related website that is intended to enable the readers to have hands-on experience of the concepts described in the book using representatives of the various programming styles that exist within the state of the practice.

This is done by having a single example implemented in all these languages. Some of these implementations already exist on the website, some have just "placeholders" as reference to a site. Most allow downloading the software, and some using the "software as a service" model in a cloud. Six languages are already referred on the site, more will be added soon for example IBM's Websphere Business Events.

The website is being prepared by a group of students, and is now exposed for public review.

This is a first version, probably much to improve. So comments are appreciated.

This is a link to the website. The website is hosted in the book section if the EPTS website.