Saturday, September 22, 2012

The semantic overload of derived events

The term "derived events" is frequently used in the event processing terminology.  Here is an example taken from the Gigaspaces Blog of using the term "derived event".    Recent discussion with somebody who is just learning the event processing area, made me realized that we overload the term "derived event".  On one hand we define event as "something that happens", and say that a derived event is an event, thus one may assume that derived events also happen.    However, the way this term is typically used has some semantic overloading.   There are actually different types of derived events.

One type of derived event is really an event that we did not observe directly, but concluded that it happened by observing other events.  Such cases is the case of fraud detection, money laundering detection and system problem diagnostics.   

Another type of derived event is a notification.   We are doing a calculation based on events and the result is notified to some person of application.  For example: derived event that calculates the highway fees, based on exit and entry events on the highway, and rate calculated by load on the highway.   This is a derived event, however - it does not really stands for something that happens, the happening here is a notification to the driver how much the fees are,  there are many derived events of this type.

The question is whether we should make distinction between the two cases. From semantic point of view they are clearly distinct.    From execution model and language point of view -- they are indistinguishable; both take events as input, apply some assertions and functions over a collection of event, and create a structured message sent on some channel.    From semantic point of view there is a difference,  the question is from pragmatic point of view, is this distinction important for somebody that takes any role in the life-cycle of the application.   More -later.


Anonymous said...

The difference should be distinguishable both semantically and computationally. In the case of a derived event, the computation should take a set of events as an input and produce a new event (and not a message). The other type, not being an event (an event's consequent?) may produce a message.

Anonymous said...

Hi Opher,

to thoughts on that topic based on my project experience

(1) From a business perspective the semantics make a huge difference since the meaning of the actual event is something completely different
(2) From a technical perspective, I personally like to keep it the same as long as the functional aspects do not require a different handling which is commonly the result of the different semantics (e.g. how do you have to treat the event regarding to persistence requirements, etc.)

One common approach for this is a generic implementation with an additional parameter for further processing in the event if there is a need for.

Best regards,


PatternStorm said...

Hello Opher, it may also be understood as follows: there's no derived event but a detection of a composite event which is an entry event followed by an exit event of the same vehicle. The calculation of the highway fee, as well as the delivery to the vehicle of that fee, is the "business" logic associated to the composite event. This view does not see the delivery of the fee as an event but as part of the business logic. If we were to create an event that represented the composite event for other "listeners" to react then we would have a derived event. So a derived event represents a composite event with a single event...

Jeff Adkins said...

Couldn't one make the case in the toll road example that there are two events on exiting the toll road?

One is the sensor stating "Car arrived at Booth exiting" which has minimal information car, time, booth, etc.

However, there is a second event that indicates that the toll trip has completed. "Trip Completed" which has a richer set of information car, time entering, time exiting, booth entered, booth exited, mileage, ave speed, fare.

There are two nouns in a sense: the car and the trip. The trip is definitely derived (and bounded) by the car's two events of entering and exiting. There is a specific purpose for the car being at those two points in space-time. So, there is a relationship between these two points which is called a trip that is implicit. We can't directly sense a trip, but we do know that when a car enters and subsequently exits the toll road, that a trip has occurred. To me, it seems that this fits your first type of derive. And I can have a more rich set of information on a derived event that on the events used in the deriving showing that the whole is greater than the sum of the parts.

In addition, we could enrich this trip noun even further if we had sensors that could identify a car (such as by license plate) as it entered then we could observe such things as entering and leaving rest areas or travel plazas derving the time spent at the stops which in turn would improve the average speed calculation.

The could also have a higher level of noun called the traveller that contained a collection of trips. To see how much reuse the toll road was given or if it made sense to implement a frequent traveller program or where to invest in plazas.

My point is that the noun (the thing that changes state) is in the eye of the beholder and there are implicit relationships that once found, can be a game-changer.