On Revision

Saturday morning, and I am spending some spare time (well -- ignoring my huge to-do list..) in reading the autobiography of Shmuel Tamir, who has probably been the most influential lawyer in Israel, as well as a political leader whom I always had great respect to (I don't admire people).

Today I would like to write about the notion of "revision" and relate it to event processing.
This is inspired, but not a direct response to a thread of discussion started by Peter Lin in the complexevents forum, under the name: mutability and aggregation.

Revision is somewhat different from modification; in modifications fact are modified, in revision they are revised. For example: if John Smith moves from the USA to Canada, then the facts about John Smith are modified, while if, by mistake it was recorded that John Smith lives in USA, where in reality he always lived in Canada, this is correction of recording mistake. Some people may wander what is the importance to make distinctions between the two ?

The first use of "revision" that I came about was in AI, talking about "non monotonic logic", the rationale is that using "classic logic" one can reason about the universe just if there is perfect knowledge, so the example used is that although birds typically can fly, however there are some exceptions -- Penguin does not fly, Ostrich does not fly, bird with broken wings cannot fly etc..
Let's say that Tweety is a bird and we don't know anything else about it, according to classic logic we cannot say whether it flies, however, according to the various non monotonic logics, we can say that since birds typically fly, we can assume for any practical purposes that Tweety flies, as long as we are ready to withdraw from this assumption when new information (such as: Tweety is a Penguin) becomes available, in that case we may need to retract all the assertions that were inferred directly or indirectly from the revised assertion.

Later in life, I have worked on temporal databases; one of the motivations of temporal databases have been to issue "as-of" queries, meaning -- looking what was known from a viewpoint of a certain time point in the past. For example -- if we investigate possible malpractice of a physician (I heard that the national sport of Americans is to sue their physicians) then in order to determine whether a physician made a reasonable decision we need to know what was the information available to the physician at the time that he made the decision. In order to achieve that facts cannot be deleted or changed, but we need an "append only" database, the distinction between "modification" to "revision" is important for the decision analysis, there may be a difference between -- the fever was high in the next measurement, or if the fever was high also in the measurement before the decision, but it was reported wrong and this information has been revised later. Eleven years ago I have co-edited a book about temporal databases which (among other things) discuss these issues.

Now, something about revisions and event processing. Recall that an event is something that happens, and it is reported to an event processing system using its projection which is also known as event (sometimes: event object, event message). An event that happens in reality cannot be modified or deleted, if it reflects something that happened. However, since when go to the projection on the processing system, again, if we assume that the knowledge is not perfect then we can have several cases of revision:


1. The event really did not happen, but it was reported by mistake that it happened, and the mistake was realized later.
2. The event happened, but it was not reported, and this was realized later.
3. The event both happened and reported, but some information associated with the event (reported through the event's payload) had wrong value due to error that was corrected later.

I'll post soon a continuation that discusses the implications of revisions on the processing of events. More - Later

Announcing the fifth EPTS Event Processing Symposium

Today, in the EPTS members monthly report, we made first announcement on the fifth EPTS Event Processing Symposium.

This annual meeting will take place this year in Trento, Italy, and planned for September 21-23,
The host and local arrangements coordinator is the EPTS member, Themis Palpanas from University of Trento.

We have decided to do this meeting in Europe this time; All of the four meetings that were took place so far were located in the USA, however around half of the members, and many of the active people in EPTS (including myself) are in the other side of the ocean (actually looking at the list of people who volunteered to be EPTS workgroup leaders, I think that more than half are outside the USA). It may even be true that most of the EP community is located in Europe (Israel, from geographical point of view is located in Asia, but typically has stronger ties to Europe).

The plan is to concentrate much of the symposium around report and discussion of the current EPTS working groups, which are the main activity for 2009. Also this will be an opportunity to start the discussion around the "grand challenges" for the EP area. We'll probably have again some invited speakers (this time from Europe), and some panels. More details -- later.

PS. I have written in one of the previous postings about Twitter, there is now also a CEP group on Twibe (a twitter application), set by Richard Tibbetts. I think that it posts the twitter messages sent by the group members and include "CEP" inside... Enjoy!

Off Topic: Holocaust memorial day, my father, and King Alfred

Today I will take some time to write on an off-topic; in Israel we have a memorial day for the holocaust. I am a second generation survivor; my father (who passed away 11 years ago) was the only survivor of a big family, he started the world world II in the Lodz Ghetto where he worked in a kitchen, either the one shown in the picture above, or one other, transferred to Auschwitz, where he was selected to live, while most of his family were selected to die, and kept moving in various work camps, his final place was in Dachau, and at the end of the war, he also survived the Death March in Tyrol mountains that is documented here and in this picture.




I learned about his history during this period over the years bit by bit, since my father almost never talked about this period in his life, when I asked him once, he told me the following story instead of telling me about himself, the story of King Alfred.



King Alfred has escaped due to some revolt and was hiding in a farm, when the rebelling soldiers looking for him got there, the farmer hid the King below a big pile of straw; the soldiers started to look at the straw, they nearly removed all of it, and then decided to move on. At later time the King succeeded to overcome the mutiny and returned to his throne. At some point the farmer came to visit him and he said to him -- you save my life, I can give you whatever you wish, the farmer said: I am a modest person, don't need anything, have one question to you, what did you feel when the soldiers almost got all the straw removed. The king has shown an angry face and said: this is a very rude question, hang this man immediately. The farmer was about to be hanged, a rope was already tightened to his throat, and then the king said: stop, now you see what I felt.

(This is actually what my father wrote in a letter to a relative that found him in the survivals list and sent him a letter asked him to describe what happened to him in the war).

Only recently I learned that my father has documented the names of all his family members in Yad Vashem, that scanned all their archive to the web a few years ago, here is the webpage where my grandfather is documented which includes my father's scanned handwriting,

Back to professional Blogging -- later this week.

On Event Processing related Books

The amount of books related to event processing is growing, and in the next year or so there will be several books added to the already existing one, looking at event processing from different angles.

First let's review the existing books.

"The power of Events" by David Luckham was the pioneering book in this area and became an icon by its own right, the book was published in 2002, and influenced the thinking in this area, including the name coined "complex event processing" that is in use as a name being used for various products in this area.

The book "Distributed Event-Based Systems" by three members of the research community: Gero Mühl, Ludger Fiege, and Peter Pietzuch. It should be noted that the "DEBS" community which concentrated around pub/sub and distributed middleware, has joined forces with the community that deal with event processing languages, architectures and execution models to form the event processing community.
This book published in 2006 deals mostly with the infrastructure that enables event-based interaction (such as: pub/sub) and is a good textbook on the infrastructure topics.
The book "Event-Driven Architecture - how SOA enables the Real-Time Enterprise" is a new book by Hugh Taylor, Angela Yochem, Les Phillips, and Frank Martinez available also as a Kindle book. I have not read it yet, but from the title it seems to look at EDA as a SOA pattern. Will add it to my next periodic Amazon order.

Now - to future books. Amazon also shows a book that has not been released yet, by Sharma Chakravarty one of the veterans of active databases and Qingchun Jiang. The book is entitled: "Stream Data Processing: A Quality of Service Perspective", and has "Complex Event Processing" is its sub-title, seems to be a monograph about Sharma's approach to unify stream and complex event processing that he presented in the DEBS 2008.
Annika Hinze and Alex Buchmann, again, two persons from the academia, well known in the community. Their intended book is a collection of articles, entitled: "Handbook of Research on Advanced Distributed Event-Based Systems, Publish/Subscribe and Message Filtering Technologies", and is geared towards the research community. I don't know the schedule.

Another forthcoming book by well known figures in our community: Roy Schulte and Mani Chandy is entitled: "Event Processing: Designing IT Systems for Agile Companies".

This book will provide business oriented view of event processing and its relations with various part of the enterprise architecture (SOA, BPM, BI).

Last but not least, the book that Peter Niblett and myself are writing entitles "Event Processing in Action", the book focuses on the building blocks of constructing event processing applications, and provides a deep dive of application building using a use case.

The fact that various publishers have taken the investment in event processing oriented book is an indication of the interest in this area. Enriching the community with several books with different viewpoints and focus area will help in both the understanding and teaching event processing concepts and facilities.