Monday, April 28, 2008

On Posets and Red Herrings

Hans Glide, one of the knowledgeable persons in the EP area, has made in his Blog couple of postings about Posets, one of them claimed that the term Poset in EP is a red herring and of interest to mathematicians only, the other one says that causality graph and causality inference are not the same.

Let's make some assertions in this area, in order to clarify (or, god forbid, confuse...):

1. The EP applications space is not monolithic, different applications may have different requirements.
2. The issue of total order vs. partial order has been blown out of proportion, since it is not a major differentiator among products. Products who lack some functionality (see below) can somehow hack it in practice.
3. The main importance of causality relation in general is in the ability to trace all causal descendents of a certain event, or conversely to trace reasons for taking some action, this is a key requirement in anything that requires auditing. This does not cover the entire space of applications, and if the application does not require tracing and auditing then the notion of causality has little value to the application.
4. On the other hand, there are applications that manage scenarios, like software verification, simulation, games etc - in which causality is an important abstraction, as indeed reasoning is done on partially ordered set (directed acyclic graph) of events.
5. There are several types of orders and several types of causality relations.
6. Order may be according to the time that the arrives "happens in reality" (as reported by the source), or according to the time it is discovered by the event processing system (by getting to a system's API). In some cases the order does not play any role, in other cases, such as time series that are far enough, the sequence in the system is assumed to be good enough, and in other cases, the "reality" order has to be preserved -- how orders are preserved, if it is at all posssible, is a topic for another discussion.
7. Causality can be explicit or implicit. Explicit causality is meta-data in the event processing system, it exists because somebody put it there (maybe using mining techniques) saying that if Event E1 happens, than event E2 also happens, and we can assume that E2 happened even if there is no explicit indication that E2 happened. Implicit (inferred) causality is there since E2 is an output to some computation that E1 is an input to. In the class level it can be modelled, in the instance level, it is dynamic and created at run-time.

Closing statement: there is no - "one size fits all" in event processing. For each type of applications there are types of functionality that are more or less important to be supported. Better understanding of the existing types and mapping functionality to these types, is a work we are trying to achieve, by analyzing a significant number of use cases, as one of the EPTS tasks, by a large team of volunteers from all sectors (vendors, customers, academic people). We'll have more news on this work later this year.