- "simple event processing" EPAs - filter and routing,
- "mediated event processing" EPAs - enrichment, transformation, validation
- "Complex event processing" EPAs - pattern detection
- "intelligent event processing" EPAs - prediction, decisions...
The common denominator: each of them receives events as input, emits events as output and does a single type of function.
I find this type of abstraction both very easy to explain people how EP systems work, and also basis for architecture. The EPN routing can be done by standard middleware, or in a stand-alone mode. Other terminology issues raised by David Luckham is the relationships to the "actor model" and to "engines".
The actor model is a model that helps reasoning about concurrency, while agents in AI are autonomous goal-driven artifacts. These are orthogonal terms, of course. In the context of EPA - when looking at EPAs as an executable network, we can look at each EPA as an actor and apply actor models.
Last but not least -- relationships of EPAs to engines -- an EPA is a software artifcat, it can be an instance of an engine, it can be some software that contains an engine, and it can be hard-coded program, as long as it complies with the EPA definition. In a future world, with inter-operability (and perhaps also language) standards, we'll be able to run (and maybe to self-select) multiple engines for the same EPN, residing in different EPAs.
More about EPA types -- later.
3 comments:
Hi,
I got to this post by you (Opher) after being interested in EPN and EPAs and searching for this topic on Google. Interesting is that just a few days ago we were talking about a similar subject by email. Our specific subject was recursion in event processing. Your point was that this is supported by loops in the event processing network (EPN). However, I was not convinced that recursion is not part of the event processing agent (EPA) (depends on its expressive power). Researching more on the subject I started to think of reasons for changing this architecture (better integration of EPA and EPN, declarative dynamic workflow instead of fixed structure, etc.) and I got to this post by you (Opher :), as the first link on Google when one searches for "event processing agent".
First of all, although I don't agree with fixed workflows, I agree with the overall idea of an Event Processing World/Web of agents (a specific case can be a fixed network, but it shouldn't be limited to that). There is definitely value in EPW by having separate processing entities in the form of agents communicating in a common environment (the world). This makes sense in the Semantic Web area where are distributed "namespaces" and agents in the entire Web.
Second, I see CEP agents as full agents with enough expressive power so they can compute any kind of complex events.
Regarding the hierarchy of EPAs above, I have a few remarks:
-"simple event processing" EPAs - filter and routing,
Filter means selection with some condition on the original events, while routing is part of the event processing network (EPN).
-"mediated event processing" EPAs - enrichment, transformation, validation
“Enritchment” means using the internal database to enrich the input events with some extra information; transformation is both selection on some condition on the arguments and projection of some arguments of the original events. Validation is selection on some condition and maybe a mechanism of exceptions and alarms.
I'm not sure I can see the distinction between "simple event processing" EPAs and "mediated event processing" EPAs.
(to continue)
Hi,
I got to this post by you (Opher) after being interested in EPN and EPAs and searching for this topic on Google. Interesting is that just a few days ago we were talking about a similar subject by email. Our specific subject was recursion in event processing. Your point was that this is supported by loops in the event processing network (EPN). However, I was not convinced that recursion is not part of the event processing agent (EPA) (depends on its expressive power). Researching more on the subject I started to think of reasons for changing this architecture (better integration of EPA and EPN, declarative dynamic workflow instead of fixed structure, etc.) and I got to this post by you (Opher :), as the first link on Google when one searches for "event processing agent".
First of all, although I don't agree with fixed workflows, I agree with the overall idea of an Event Processing World/Web of agents (a specific case can be a fixed network, but it shouldn't be limited to that). There is definitely value in EPW by having separate processing entities in the form of agents communicating in a common environment (the world). This makes sense in the Semantic Web area where are distributed "namespaces" and agents in the entire Web.
Second, I see CEP agents as full agents with enough expressive power so they can compute any kind of complex events.
Regarding the hierarchy of EPAs above, I have a few remarks:
-"simple event processing" EPAs - filter and routing,
Filter means selection with some condition on the original events, while routing is part of the event processing network (EPN).
-"mediated event processing" EPAs - enrichment, transformation, validation
“Enritchment” means using the internal database to enrich the input events with some extra information; transformation is both selection on some condition on the arguments and projection of some arguments of the original events. Validation is selection on some condition and maybe a mechanism of exceptions and alarms.
I'm not sure I can see the distinction between "simple event processing" EPAs and "mediated event processing" EPAs.
(to continue - I found out that the blog doesn't allow me to post the whole comment after I wrote it)
-"Complex event processing" EPAs - pattern detection
EPA pattern detection is detection of sequences of events and other patterns (based on time, filters with static conditions or conditions from the database, etc.)
Up to here EPAs were definitely "stateless". However, “recursion” presumes that agents are stateful, or, at least, have access to their outputs and compute some extra events or a fixpoint of its own derived events. That is, the relations that define new complex events are recursive. The example that we discussed in the email is:
reach_event(X,Y) <- edge_event(X,Y).
reach_event(X,Y) <- reach_event(X,Z) ’seq’ edge_event(Z,Y).
I think this still follows the definition above:
"Complex Event Processing Agents is a software artifact that receives an event cloud or stream or collection of events or a single event, does some computation on these event, and produce one or more events as an output."
However, this might also be a different category or EPAs:
-"Stateful Complex event processing" EPAs or
-"Recursive Complex event processing" EPAs
Personally, I prefer an expressivity hierarchy in the class of "Complex event processing" EPAs. In logic programming, recursive programs are a completely different category of programs then non-recursive programs. Moreover, when one has negations (classical and default negations) the hierarchy becomes much more complex, e.g., stratified programs, non stratified programs, locally-stratified programs and not locally-stratified programs, with diverse logic semantics: well-founded semantics, stable models, etc.
-"intelligent event processing" EPAs - prediction, decisions...
Just to finish the hierarchy, I think that the last category is just about integration of CEP with other semantic constructs: "prediction" IEP EPAs are CEP EPAs with underlying meaning of events (semantics like in the Semantic Web and knowledge representation areas), while "decision" refers to integration with event-actions rules (up to my understanding). The point is that this last category is regardless to the issues discussed above.
I also understand your statement that "this type of abstraction both very easy to explain people how EP systems work, and also basis for architecture. The EPN routing can be done by standard middleware...".
Finally, the point of this comment is to see if the issues mentioned above are valuable or not, and that I believe in a better overall integration of the whole architecture for complex event processing and maybe in a different hierarchy of EPAs.
I hope this post also helps other people interested in the same problems discussed above.
Paul Fodor.
Post a Comment