This is a blog describing some thoughts about issues related to event processing and thoughts related to my current role. It is written by Opher Etzion and reflects the author's own opinions
Sunday, December 1, 2013
More on event processing and mobile devices
Friday, August 30, 2013
New market research on the event processing market by Markets&Markets
It seems that there is a new comprehensive market research on the event processing market in the years 2013-2018 by Markets and Markets. I don't have the market research itself (it is quite expensive), but the site gives some details, according to the report, Markets&Markets forecast that the "CEP market" is expected to grow from $764.5 million in 2013 to $3,322.0 million in 2018. I wonder what this figures represent, it seems that this is beyond the accumulative sales of event processing platforms.
BFSI: algorithmic trading, electronic transaction monitoring, dynamic pretrade analytics, data enrichment, fraud detection, governance, risk and compliance (GRC);
Transportation and Logistics: asset management and predictive scheduling and toll system management; healthcare: self-service proactive monitoring and alerting and governance, risk and compliance (GRC);
Telecommunication: mobile billboards, revenue assurance, network infrastructure monitoring and predictive CDR assessment;
Retail: inventory optimization, shoplifting detection and real-time marketing and customer engagement;
Energy and utilities: oil and gas operation management and nuclear crisis and smart grid energy management;
Manufacturing: shop floor automation and operational failure detection, infrastructure management and supply chain optimization;
Government, defense and aerospace: Intelligence and Security, emergency response services and geo-fencing and geospatial analysis;
Others: includes education and research
Monday, March 25, 2013
Event processing at the core of automated commerce systems
Wednesday, January 30, 2013
Event processing with Storm-Cassandra from health market science
Monday, January 21, 2013
DEBS 2013 grand challenge was announced
Thursday, January 10, 2013
From Australia - 2013: The Year of the Internet of Things
Saturday, November 10, 2012
On IBM scientific accomplishment
Sunday, May 27, 2012
Event processing and mobile devices
- Mobile device can be a producer of events -- this may be both implicit events, using the mobile devise as a sensor and tracking the location of the mobile device over time or explicit events, where a human use mobile devise to report events (such as: traffic jams, completion of tasks, start of the work day)
- Mobile device can also serve as consumer of events -- subscription to either raw or derived events can be manifested within notification in form of Email, SMS, or mobile application. It seems that most uses of mobile devises are the consumer and producer roles. There are also some more sophisticated possible uses, discussed next.
- One can build event processing agent logic using mobile devise, but it is more likely that the use in this case will be to use predefined templates in order to define specific instances of EPA. For example, if the application is subscription to events in my bank account, I might get a template giving me an opportunity to subscribe when my accounts reached a balance of more than X or less than Y, I received a deposit of more than Z, or that the total withdrawals for a single days are more than W. Using the mobile device I can chose any of them and give value to the X, Y, Z, W variables.
- A dashboard showing the real-time state considering the streaming events can be also displayed as a mobile application
- Last but not least -- An EPA can reside on the mobile device itself, collection data and doing some filtering or aggregation.
Friday, May 4, 2012
On robots for the elderly
Tuesday, May 3, 2011
Event processing for oil and gas -- Microsoft Streaminsight presentation
Thursday, July 8, 2010
Announcing the EPTS awards


Monday, June 21, 2010
On students' projects in the event processing course
The semester has come to its end, and the event processing course in the Technion had the last meeting today; the students still has to finish the projects, and they have some time to do it, but today they have presented a preview of the project, here is a brief summary. They were given a choice to do research oriented project or implementation oriented project, those who chose implementation oriented project got some directions on the size (quantities of consumers, producers, patterns, aggregations, filters etc...), and could chose both the application they want to work on (and they'll have to test it with simulated data) and the tool they use in order to implement. - Team no. 1 -- chose to work on a research problem of interval-oriented event processing and defined a collection of operators and patterns required to support interval oriented.
- Team no. 2 -- chose to work on "smart traffic control", simulating traffic lights, traffic jams, accidents and events like concerts etc... They decided to hard-code their application in Java, since they know Java well, and it is more fun doing it this way :-).
- Team no. 3 - chose to work on "smart home" and mainly concentrate on the security aspect of home automation, they are also Java fans, but chose to use Esper in conjunction with Java.
- Team no. 4 -- chose to work on control of people who are abusing chat rooms in various ways (like using dirty words), and managing complains. They have chose to work in Streambase, and done all implementation (so far) using the graphical tool of Streambase. We again see that some people like graphical programming and some people prefer old-style imperative programming.
- Team no. 5 -- chose to work on tracking bus driver performance, both for safety issues (speed), and accuracy (skipping bus stations). They have organized late, so have not started the implementation phase yet, but they said that they wish to do the project using Apama.
The projects are due in August, and then I'll write more, if I'll have any interesting insights.
Wednesday, March 10, 2010
Revisiting race condition with FFD example
In the past I have written about race conditions and this triggered some responses. We recently realized that in the example we created for the EPIA book (the Fast Flower Delivery and has got already around ten different implementations, six of them can be viewed on the book's webpage, some more will be added) there is an case that if will not be handled carefully may yield wrong results due to race conditions. Here is the case:Tuesday, March 9, 2010
On new event processing course and some ways to explain what event processing is
This is the Computer Science building at the Technion; yesterday I have started to teach an event processing course there (in the previous semester I have taught in the Information Systems Engineering program, which is my academic home for many years), the course is given as an "advanced topics in computer science" course, and the students that have shown up were mostly graduate students from CS as well of EE (there are software people in EE also). Unlike the previous semester in which the students projects evolved around validating the solutions that different vendors put for the EPIA book's website, this time I would like to concentrate in the project level about the view of those building event processing platform (this is a slightly different view). The first class is always an introduction, since the students don't have a clue about what event processing is, thus I am starting with some examples about what is it used for.
Sunday, December 20, 2009
On common misconceptions about event processing - the single application misconception
We start the introduction chapter for the EPIA book, by stating: Some people say that event processing is the next big thing; some people say that event processing is old hat and there is nothing really new in it. Both groups may be right to a certain extent. As with any field that is relatively new there is some fog around it: some of the fog stems from misconceptions, some from confusing messages by vendors and analysts, and some arises because of a lack of standards, a lack of agreement on terms, and a lack of understanding about some of the basic issues. In the book we don't really talk about the misconceptions, but I think it is a good topic towards the end of 2009 to dedicate some postings towards the major misconceptions.
I'll start with misconception number 1: Event processing is a single-industry (some even say single-application) technology, and event processing software cannot generalize beyond this single industry/application.
The industry is, of course, capital markets, and the application is algorithmic trading
The diagram below is taken from the ebizQ customers survey (two years ago) about what are the business problems that they expect to solve with event processing, and the result is 9% indicated algorithmic trading.
This misconception is originated from the fact that the capital market industry has indeed been the early adopter of event processing software, and served as a proof of concept for the rest of the industries, there are indeed some vendors that focus mainly around this type of application, however, this does not show the entire picture. From the IBM experience I know of customers in various industries, most are not in the capital market area. Getting to the material collected by the EPTS use case work group (the material is on the EPTS members internal site, available to EPTS members) I find quite a lot of examples of systems working in production or being developed from variety of domains, here are some samples:- Border security radiation detection (Eventzero)
- Mobile asset geofence (Rulecore)
- Logistic and scheduling application (Starview)
- Unauthorized use of heavy machinery (Rulecore)
- Hospital patient and asset tracking (IBM)
- Activity monitoring for taxing and fraud detection (IBM)
- Intelligent CRM in banking (TIBCO)
- EDA and asynchronous BPM in retail (TIBCO)
- Situation awareness in energy utilities (TIBCO)
- Situation awareness in airlines (TIBCO)
- Reduce cost in injection therapy (IBM)
- Next generation navigation (CITT)
- Real-time management of hazardous materials (Oracle)
- Finding anomalies in point of sales in retail stores (CA)
- Elderly behavior monitoring (U. of Munich)
Tuesday, November 10, 2009
On the Event-Driven Architecture book

Monday, September 28, 2009
On Adaptive services

Chameleon is an adaptive animal, it can adapt it color to the environment. In the next few weeks my main task will be to complete a proposal for EU project that deals with adaptive services. I have (mistakenly?) agreed to coordinate a consortium that creates a proposal. We had a meeting in Trento following the EPTS event processing symposium. EU project has a benefit of getting funding for research that can explore more advanced topics then can be funded by commercial corporates, and also provide an opportunity to collaborate with some very good people both in industry and academia. The down side is that there should be much investment in the proposal, since it became extremely competitive.
The proposal is about the idea of adaptive services -- which means in plain words that the behavior of the system should be adapted to (unexpected?) events. This adaptation may require human interaction (e.g. modifying medical treatment protocols), or self-adaptation (e.g. change of emergency handling protocol on the fly). The research challenges here are in area of event processing -- having more advanced features that are beyond the state-of-the-art; adaptation, modeling and methodology of how to build such systems. More about this topic - later.
Tuesday, July 28, 2009
On social networks and event processing

Today I noticed that the number of my LinkedIn contacts is now a round number 0f 600, unfortunately two of these 600, Klaus Ditrtich and Shlomit Zak have passed away, but their LinkedIn records are still alive, I hope that the rest of the 598 are alive and well. I was introduced to LinkedIn 3 or 4 years ago, when I received a LinkedIn invitation from Mark Palmer, who was my first contact, and accumulated the rest of the links over the years. Part of them invited me, and part of them were invited by me, I have there classmates from high school, colleagues from all places I worked in, students I taught or supervised, some friends, and of course, the members of the event processing community. LinkedIn was my first social network, currently I even don't know exactly how many social networks I am member of, but LinkedIn is the only one that I am active in. So I'll take this opportunity to say a few things about how event processing can become part of social network platforms.
There is a lot of buzz recently about Twitter events, which is also one of the Web 2.0 family, but I'll take the LinkedIn example now, since it has some structure, which creates various event types. While there is applications that attempt to use Twitter events to get information about the stock market, in social networks the events are about individuals; some of the interesting event patterns that may be obtained about individuals:
- A person got more than 3 recommendation during the last week -- probably looking for a job now, this is an information that can be of interest for various people, such as head hunters, or enterprises seeking for employees.
- A certain amount of contacts joined a certain group -- may be an indication about a group of interest
- LinkedIn has microblogging of 14o characters like Twitter -- various things can be obtained from this microblogging.
- Postings that people make on groups may indicate something about a person.
- Level of activity in groups may also indicate something about that group.
Monday, July 13, 2009
On the vision for smarter cities

The IBM Institute of Business Value has recently published its vision of smarter cities, which is part of the "Smarter Planet" vision on which I have written before. As noted before, the smarter planet vision is based on the fact that the world will be instrumented, interconnected and intelligent. Event processing is the backbone of smarter cities. As smarter cities is been labeled "system of systems", the integration and communication between the different systems is done using events, and the event processing filters, transforms and routes the event among systems. It also crates derived events from one or more systems that may be used by other systems. Event processing technologies have two roles: In the macro level, it is a glue between the systems and is part of the intelligent part of each individual system. I'll write more about some individual systems inside the system of systems. I am trying to get a smarter cities keynote talk in the 5th EPTS event processing symposium --- stay tuned, details about the symposium (Trento, September 21-23) are coming soon.
Tuesday, July 7, 2009
Live from DEBS 2009 - Keynote Address by John Bates

In my role as the industrial track chair of DEBS 2009, I have been asked to invite an industrial keynote speaker, and this year I chose John Bates, the General Manager of the Apama Division of Progress Software, and a person that made a transition from being a faculty member in Cambridge University, leading one of the first start-ups in this area, and now managing a division in a larger company. This was certainly a good choice; John outlined the assumptions he had as a researcher, and the confrontation between these assumptions and the reality that forced them to find a low hanging fruit, until the bigger market will be mature enough for the ideas for event processing, and they found trading in capital marketing as this low hanging fruit, and the early adopters of such technologies. However, the maturity in thinking about this area now takes it to more areas. As for John's prediction for what will happen in the event processing market in the next five years, he mentioned the following points:
- Event Processing will be used for tracking everything -- car, plane, bag, package, ship
- EDA -- will achieve wider adoption -- federating services in the enterprise, providing agile enterprise nervous system; The next wave of SOA.
- Event processing everywhere -- EP in the cloud.
- Event Processing will be part of bigger event-driven business process management market - BPM, rules, event processing are going to merge.
- Event processing will be embedded in many vertical applications.
More about further sessions in DEBS 2009 - later










