Latency might be subjective

Last week I have been in a short visit to England, served as external examiner in a PhD exam (called by the locals: "Viva"). The exam was strange in the sense that it was the first PhD exam I have ever attended where the advisor was not present (actually he was not invited, I guess it is a matter of culture).   The dissertation that was submitted by Jenny Li dealt with performance metrics and benchmark framework for event processing system.  One of the ideas raised in this dissertation is that the measured latency may be subjective in what it means.   It took me some time to understand the idea, so I'll explain it through example.  

Let's assume that the pattern that we are looking for is a sequence of four events, of types E1, E2, E3, E4.

The two common metrics associated with latency are:

Measurement starts at E4, since the consumer expects to see results only when the last event that closes the loop occurs.

Measurement start at any event occurrence, and ends when the system finishes processing the individual event, which can be merely storing it in the buffer.

Note that the first metric is biased towards eager evaluation, doing the minimal work at the end, and preparing sub-patterns, while the second metric is more balanced. 

The proposed metric is -- start measuring in the event most significant to the consumer, and end at the end of the processing of the pattern.   Let's say that the most significant event to the consumer is E2, then the latency starts at the occurrence of E2, and ends either when there is a match of a pattern, or when the event E2 can be discarded since there is no match.   This can be applicable in cases that all events in the sequence typically happen (e.g. when they are time series events) in relatively fixed differences.   This is an interesting metric, we asked the student to define definite criteria for when it is an applicable metric and when it is not.   

On decision latency

James Taylor wrote in his Blog an article on decision latency, the illustration above is taken from his article. 

The latency for getting a decision consists of three time intervals:

The Capture latency is the time taken from the event occurrence in the universe until it is detected (captured) by the processing system

The Analysis latency is the time taken from the time that the event is detected by the system, until the time it is  processed and creates the derived event/data that triggers an action, and the Decision latency has two parts - which I think should be distinct,  the time to make the decision how to react, and the time it takes to react.

These four type of latency are actually orthogonal.  For the Capture latency what is required is an infrastructure of instrumentation, sensors and communication; for the Analysis latency -- the event processing system latency, and possibly additional analysis required by querying historical databases,  for the decision latency --  the decision tools (either a decision management system or optimization system), and for the action latency -- the speed in which it can be applied. 

Why the decision latency is important? -- here we are getting back to the real-time issue.  In some cases the time is critical for decisions,  an example that we saw a few weeks ago is in the Israeli anti-rockets systems

known as "Iron Dome" (Kipat Barzel).  The system detects that a rocket was launched, and then it needs to analyze its course and determine whether it is going to hit a populated area, in order to decide whether it is cost-effective to send a missile to hit the rocket (this decision is needed since a missile is very expensive), and if yes,  determine when and where the missile will intercept the rocket and launch the missile at the right time to the right direction.  Latency in all the four phases is critical., and the success of this system saved many lives.    

Decision latency is not only related to defense systems,  it exists in different other areas - healthcare is certainly one of them, but also business.  The time scale does not have to be seconds,   decision latency can span minutes or hours,  

We can look at the value of decision as function of its latency,  and can see that there are soft real-time cases which the value of decision goes gradually to zero, Firm real-time where it goes to zero at a certain deadline.

Hard essential in which it goes to a certain penalty (such as missing SLA), and hard critical where it is going to a minus infinity (a disaster case), the Iron Dome case is of that type.  

Of course, there are no miracles, and the cost of a system grows when bounded latency need to be guaranteed (for example cost of extremely robust communication).  For each type of event-based system there is a need to calculate the trade-off between the value of latency and the cost. 

On latency in event processing network

Packing, on my way to a family vacation in Anatalya, Turkey.

Following an interesting discussion yesterday about performance metrics -- it turns out that definition of latency in event processing network is quite tricky. The reason is that an event can move in multiple paths over the network, in some it is filterred out, in some it just getting into internal state of agent since it does not complete a pattern, and in some it may complete a pattern and trigger reaction, thus there are various ways to define the metrics here. This is important since optimization has to take into account the goal function - what is being optimized. More discussion on this area -- after I'll return. I'll be out of touch for a week.

On Real-time, Right-time, latency, throughput and other time-oriented measurements

The illustration shows different types of "real time" types. This posting was inspired by a comment on a previous posting - trying to do some order in several notions of time. First, the term real-time is frequently used in conjunction with event processing. The popular belief is that real-time = very fast, but this is not really what real-time is about. real-time can be thought of a deadline accompanied by a utility function that designates the damage from missing the deadline. In this illustration there are four type of real-time:

(a). Soft Real-Time: there is a sense to react after the deadline, but the utility decreases (maybe fast) and at some point gets to zero - no use to do it at that point, but no damage.
(b). Firm Real-Time: The utility go immediately to zero when the deadline is missed - no use to do it after the deadline, but no damage.
(c). Hard Essential: Missing the deadline - the utility function goes to a constant negative value; there is a constant penalty.
(d). Hard Critical: Missing the deadline - the utility function goes immediately to "minus infinity", means: a catastrophe will happen.

One can, of course, define the real-time utility function differently, and create more variations.

So - Real-time is not about fast, but about missing the dead-line. The linkage is there, if the dead-line is very short (need to react within 1/1000 of a sec), but many dead-lines are longer than that -- seconds, minutes, hours or days - depends on what is needed to react to - e.g. the contract that we have with our local electricity company says that when a problem is reported, they should start fixing it within 2 hours; the deadline for delivery of Pizza (otherwise it is free of charge) in one of our local delivery centers is 40 minutes - so most the world typically does not work in milliseconds.

When talking about "quality of service" measurements they can be either statistical: in 90 percent of the cases, the deadline should be reached, or individual: for all cases the deadline should be reached. Typically there are different scheduling strategies to achieve each of them - for the individual case it is important to have a consistent deterministic reaction, and this is a source of the various implementations of Real-Time Java - since Java is non deterministic by its nature of garbage collection that happens in undetermined times and lasts for undetermined duration, which may work for the statistical case, but not for the individual case. The Real-Time Java does not stand for Java which runs much faster, but for Java in which the garbage collection differences are smoothed, thus its reaction is (more) deterministic.

What other measurements are there ? in event processing - latency can be measured by the end-to-end (from the time that the event happens in reality to the action being taken in reality), can be related only to the event processing part (from the producer sends the event, until the time that the consumer(s) receive the consequences of this event), and can relate to a specific function (agent) in the event processing network, so when latency is mentioned - it should be defined - what is really being measured. The deadline typically refers to time constraint of the latency.

The term throughput designates the amount of events that the system can get as an input in a given time interval (1 second, 1 minute etc...), throughput does not necessarily entails that all these events are handled within this time interval - it may be the case that the events are put into a buffer, and all of them are handled within the next 6 hours - so in this case, the throughput is mainly determine by the capacity of input channels and buffers and not in the speed of processing. Of course, there are applications that require high throughput together with individual hard real-time constraint on each event.

The "Right-time" to react is determined by these time constraints, and determine the scheduling and optimization requirements - more later.