- In the vendors world, Microsoft has announced a forthcoming product, Software AG notified that it is working on a product, and more start-ups have joined the area; the most notable acquisition this year is the acquisition of Coral8 by Aleri that was not an intuitive acquisition.
- In my own company, IBM -- besides Websphere Business Events (WBE) that was launched in 2008 and is growing rapidly in 2009 in number of customers, IBM announced three more products in this area in 2009: Infosphere Streams, Websphere Sensor Events, and EDA extension for CICS, as IBM believes in having event processing capabilities pervasive throughout its software portfolio
- The emergence of new book. The first book in this area that has made a big impact was David Luckham's "Power of events". Eight year have passed with Luckham's book as a single book in this area. In 2009 several more books have been published, most notable the book by Chandy and Schulte. Some more books are due in 2010
- Some popular magazines ran articles about event processing, one of them is International Journal of Banking Systems, and the other is IEEE Computer.
- All major analysts had special reports on event processing. Gartner has written about it before, but now made it explicit part of its "hype cycle"; Forrester made a thorough report with comparison among several products over multiple criteria.
- The major scientific conference of event processing DEBS has been endorsed by ACM and became ACM DEBS conference. The conference made a shift over the last couple of years from "pub/sub" conference to a larger event processing conference. EPTS provided two tutorials: languages and use cases
- Other event processing related workshops interacting event processing with other areas were: event-driven business process management, or event-based processing in robotics. These two topics have been discussed in the annual EPTS meeting that was held in Trento.
- it is announced that Streambase will receive the "world economic foundation" award, an indication that event processing is considered as one of the influential technologies for the world economy.
- Another winner of the same award is Twitter. This year different applications that processing Twitter events have emerged.
- The quote of the year comes from Alex Buchmann, in his Keynote address in DEBS 2010: regular programming is like drinking with a straw, this is good when the data is standing, while the data is moving, like in event processing, using the same kind of thinking is similar to use a straw to drink from a waterfall.
Wednesday, December 30, 2009
Wednesday, December 23, 2009
- The complexity may stem from the fact that we don't know what exactly we are looking for, and generally looking for anomalies in the system (e.g. trying to find security violations), thus some AI techniques have to be applied here.
- Another case is that we know what are the patterns or aggregations we are using, but they require complex computing, in term of functional capabilities.
- Another case is that the complexity is in some non-functional requirement: such as scalability in several direction (scale-up or scale-down), strict real-time performance constraints, highly distributed system etc..
- Another case of complexity is in interoperability, the need to obtain events from many producers, and use events in many consumers, which requires instrumentation/modification of a lot of legacy systems.
- Yet another case of complexity may be unreliable event sources, handling false positives and false negatives.
Sunday, December 20, 2009
In the book we don't really talk about the misconceptions, but I think it is a good topic towards the end of 2009 to dedicate some postings towards the major misconceptions.
I'll start with misconception number 1: Event processing is a single-industry (some even say single-application) technology, and event processing software cannot generalize beyond this single industry/application.
The industry is, of course, capital markets, and the application is algorithmic trading
The diagram below is taken from the ebizQ customers survey (two years ago) about what are the business problems that they expect to solve with event processing, and the result is 9% indicated algorithmic trading.
- Border security radiation detection (Eventzero)
- Mobile asset geofence (Rulecore)
- Logistic and scheduling application (Starview)
- Unauthorized use of heavy machinery (Rulecore)
- Hospital patient and asset tracking (IBM)
- Activity monitoring for taxing and fraud detection (IBM)
- Intelligent CRM in banking (TIBCO)
- EDA and asynchronous BPM in retail (TIBCO)
- Situation awareness in energy utilities (TIBCO)
- Situation awareness in airlines (TIBCO)
- Reduce cost in injection therapy (IBM)
- Next generation navigation (CITT)
- Real-time management of hazardous materials (Oracle)
- Finding anomalies in point of sales in retail stores (CA)
- Elderly behavior monitoring (U. of Munich)
Saturday, December 19, 2009
The one week school break for children ends tomorrow, and today I went with my two younger daughters to see Avatar, a very ambitious movie with a lot of advanced 3D graphics. The downside is that the film spans over 3 hours without a break -- could cut 1 hour easily, but nevertheless the graphics is very impressive, the plot is kind of paraphrase on other movies.
Friday, December 18, 2009
Thursday, December 17, 2009
Tuesday, December 15, 2009
Like natural languages, in event processing we have many languages, and languages styles (when preparing the course I am teaching now I realized that there are more styles than I've realized, next class we'll discuss it in class, showing examples from all the different products participating in the EPIA book's website).
Sunday, December 13, 2009
Friday, December 11, 2009
This is a link to the website. The website is hosted in the book section if the EPTS website.
Thursday, December 10, 2009
Saturday, December 5, 2009
Monday, November 30, 2009
We now have all chapters of the book ready in a draft form !!!, several of them are still in cleaning phase, we will send them to another set of reviewers (Manning is champions of reviews, it did a review on the outline, and three reviews during the book), and will hand out the final copy sometimes in January, so the target is to have the book out around the end of April.
Saturday, November 21, 2009
Friday, November 20, 2009
As shown in this figure there are several reasons for making one or more of these assumptions invalid.
Thursday, November 19, 2009
- Aleri (actually the CCL language originally Coral8)
- Apama (owned by Progress Software)
- Rulecore (a Sweden based company)
Tuesday, November 17, 2009
Saturday, November 14, 2009
Friday, November 13, 2009
Wednesday, November 11, 2009
- State-change view - an event is a change in the state of something and as such is reported. Its properties: a change must occur, and this change must be reported. Example: An item previously outside the range of RFID reader, is now within the range of this RFID reader.
- Happening view -- an event is anything that happens, or is contemplated as happening (the EPTS glossary definition), in this case, a change must occur, but its reporting to the system is optional, not every event according to this definition is of interest to be reported. Example: A person sending Email
- Detectable-condition view -- an event is a detectable condition that can trigger a notification, in this case a change does not have to occur, but reporting should occur. Example: A GPS devise reporting track location (note -- location may not have changed since last report. since the track driver went for lunch).
Tuesday, November 10, 2009
Monday, November 9, 2009
Sunday, November 8, 2009
- Occurrence time that occur over intervals: Events typically occur over intervals, but for computational reasons it is convenient to approximate it to a time-point, and look at events in the discrete space; however, for some events this is not an accurate thing to do, and interval-based temporal semantics should be supported, along with operations associated with them.
- Temporal properties of derived events: For raw event, we defined occurrence time as the time it occurred in reality, and detection time, as the time that the system detected its existence. What are the temporal properties of derived events? there is no unique solution to this question.
- Out-of-order events: This topic is the topic most investigated among the challenging topics, however, current solutions are based on assumptions that are sometimes problematic. This problem is about events that arrive out of order, where the event processing operation is order-sensitive.
- Uncertain events: Uncertainty whether event has happened, due to malfunction, malicious or inaccurate sources
- Inexact content of events: Similar to uncertain events, some content in the event payload including temporal and spatial properties of the events may not be accurate.
- Inexact matching between events and situations. Situations are the events that require reaction in the user's mind. This is in getting us back from the computer domain to the real-world domain. Situation is being represented as a raw or derived event, but this may be only approximation, since there may be false positives and false negatives in the transfer between the domains.
- Traceability of lineage for event or action, this gets to the notion of determination of causality. Since in some cases there are operations in the middle of the causality network outside the event processing systems boundaries (e.g. event consumer who is also event producer) causality may not be automatically determined.
- Retraction of event: ways to undo the logical effects of events, sometimes tricky or impossible, but seems to be a repeating pattern.
Wednesday, November 4, 2009
Tuesday, November 3, 2009
- Logical operator patterns: all, any, absence that designate conjunction, disjunction and negation event patterns.
- Threshold oriented patterns: count of events, average/maximum/minimum of some attribute of a collection of events has some binary relationship (e.g. > ) with a given threshold.
- Relative patterns: relative max/relative min selects the events with the minimal or maximal value for a certain attribute over a collection of events.
- Modal patterns: sometimes, always, select a collection of patterns if a certain predicate is satisfied over all/some of the events in this collection.
- Not select pattern: This is a second level modal pattern that selects events that were not selected by a certain patterns.
- Sequence pattern: A temporal pattern that denotes a conjunction of event that occur within a predefined order.
- Trend patterns: Temporal patterns that detect trend, e.g. a value of a certain attribute is consistently increasing with a context.
- Spatial distance patterns: These are similar to the threshold patterns, but relate to the distance of events from some point in space.
- Spatial relative patterns: This are similar to relative patterns, but relate to the relative distance of events from other events
- Spatiotemporal patterns: This combine temporal and spatial properties, and designate direction of movement (e.g. moving consistently north, moving towards some entity).