The event processing course that I taught in the Technion has ended (except for the make-up exam in March). This year the students' project has been different from the last couple of years (well, I am bored having the same type of project all over again). The students were asked to select a research project. My Teaching Assistant was very skeptic about the students' abilities to do such projects, but after the presentations they did in the last class summarizing what they did so far she changed her mind. I'll write about these projects after they'll be submitted (due in early March). One project is investigating the issue of compensation and revision in event processing, I have written about revisions before, both in general, and specifically related to event processing.
There are couple of motivations of why I have returned to be interested in it now.
One of them is the investigation of uncertain events, since more information may be acquired with time about events that are uncertain, a revision might be needed.
The second is the work on future events, since future events are obtained using a forecasting process, this forecasting process is not a one-time process, but can be sensitive to additional events that happen between the forecast and the occurrence of the future event, thus the forecast itself may be revised.
The issue of revision may entail the need for compensation for decisions and actions already taken.
- In some cases it is easy, example when no action was taken and it is still possible to take action
- In some cases it is impossible, if an action has been taken, and this action cannot be retracted,
- In the remaining of the cases, it might be possible, however not always cost-effective, since it might have cascading effect of compensating for large amount of actions.
After getting the students' work, I'll write again about this issue.
Long day today, I got to the office around 8AM and left around 9PM. Since we have holiday this week I am trying to condense the remaining days of the week and the result is long day with plenty of conference calls. The picture above is a glance (from below) on the IBM Haifa Lab (the pair of connected building on the right hand side of the picture), my office is in the back building (known as the "banana" due to its shape), and is not really in the nice part of the building -- the one with the view to the Haifa Bay -- well, one can have everything in life -).
I still need to complete the previous posting on revision. I gave some explanation about the concept of revision, and now I still need to discuss implementation of revision in event processing. To recall -- a revision in event processing is getting later knowledge that asserts that either a reported event did not really happen, or some information associated with the event was wrong.
Let's look at two separate cases, one in which the processing has not gone out of the event processing network, and second that the results of the processing have gone out to the "outside world".
In the first case, there may be an opportunity to revise the impact of the revised event by doing kind of undo-redo for all the event processing agents that it passes directly or indirectly. Direct ones are easy -- those that the revised event participate as an input in them, indirect is more tricky, since we need to trace the causality among events, in this case, an event that is an output of an event processing agent in which the revised event participate (relate to the same context) has a causality relation to the original event, thus, an event processing agent, in which this event participates as an input, also needs to do an undo/redo, and causality is a transitive relation, so it continues as far as the EPN arrived so far. It should be noted that the fact that there is a causality may not require a real undo/redo, take as an example that an event of type E1 designates a bid, and the event of type E2 designates the bid with maximal value arrived in a certain time interval. Let's assume that a certain bid has been revised, however, neither the revised bid, or the revising bid change the selection of E2.
The second case is that the revised event has consequences that have been sent to an external consumer, thus, it may have triggered an action, a collection of actions, or a workflow that has been carried out, and this may propagate further ("the butterfly effect"), in this case, either we can treat it as "too late" and do nothing, however, there may be a cases that it can be critical to undo/redo also the consequences, e.g. the revised event has some financial meaning. In this case we'll need to issue compensation for the triggered action, which may be impossible (the consumer does not support compensation) or difficult. I'll blog again about revising the history and its aspects at a later phase.
Saturday morning, and I am spending some spare time (well -- ignoring my huge to-do list..) in reading the autobiography of Shmuel Tamir, who has probably been the most influential lawyer in Israel, as well as a political leader whom I always had great respect to (I don't admire people).
Today I would like to write about the notion of "revision" and relate it to event processing.
This is inspired, but not a direct response to a thread of discussion started by Peter Lin in the complexevents forum, under the name: mutability and aggregation.
Revision is somewhat different from modification; in modifications fact are modified, in revision they are revised. For example: if John Smith moves from the USA to Canada, then the facts about John Smith are modified, while if, by mistake it was recorded that John Smith lives in USA, where in reality he always lived in Canada, this is correction of recording mistake. Some people may wander what is the importance to make distinctions between the two ?
The first use of "revision" that I came about was in AI, talking about "non monotonic logic", the rationale is that using "classic logic" one can reason about the universe just if there is perfect knowledge, so the example used is that although birds typically can fly, however there are some exceptions -- Penguin does not fly, Ostrich does not fly, bird with broken wings cannot fly etc..
Let's say that Tweety is a bird and we don't know anything else about it, according to classic logic we cannot say whether it flies, however, according to the various non monotonic logics, we can say that since birds typically fly, we can assume for any practical purposes that Tweety flies, as long as we are ready to withdraw from this assumption when new information (such as: Tweety is a Penguin) becomes available, in that case we may need to retract all the assertions that were inferred directly or indirectly from the revised assertion.
Later in life, I have worked on temporal databases; one of the motivations of temporal databases have been to issue "as-of" queries, meaning -- looking what was known from a viewpoint of a certain time point in the past. For example -- if we investigate possible malpractice of a physician (I heard that the national sport of Americans is to sue their physicians) then in order to determine whether a physician made a reasonable decision we need to know what was the information available to the physician at the time that he made the decision. In order to achieve that facts cannot be deleted or changed, but we need an "append only" database, the distinction between "modification" to "revision" is important for the decision analysis, there may be a difference between -- the fever was high in the next measurement, or if the fever was high also in the measurement before the decision, but it was reported wrong and this information has been revised later. Eleven years ago I have co-edited a book about temporal databases which (among other things) discuss these issues.
Now, something about revisions and event processing. Recall that an event is something that happens, and it is reported to an event processing system using its projection which is also known as event (sometimes: event object, event message). An event that happens in reality cannot be modified or deleted, if it reflects something that happened. However, since when go to the projection on the processing system, again, if we assume that the knowledge is not perfect then we can have several cases of revision:
1. The event really did not happen, but it was reported by mistake that it happened, and the mistake was realized later.
2. The event happened, but it was not reported, and this was realized later.
3. The event both happened and reported, but some information associated with the event (reported through the event's payload) had wrong value due to error that was corrected later.
I'll post soon a continuation that discusses the implications of revisions on the processing of events. More - Later
