Wednesday, July 15, 2009

More on states and event processing

During my last visit in the USA I have purchased and now reading the book "the last oracle" (nothing to do with the software company that bears this name), one of the thrillers that are based on some history and span around the world, exposing some scheme that will bring disaster on the universe. It seems that this type of books became popular recently.

Anyway, I wanted to come back to the issue of states that I have recently blogged about. Actually the term state in event processing are somewhat overloaded. A state may refer to any of
the following:

  • The internal state of a single agent -- when trying to detect a pattern (say - E1 and E2) there is a need to keep the state, e.g. the fact that E1 already arrived but E2 did not. This can be kept either by keeping the input event as an object, keeping another object somehow derived from the original event, or keeping some internal data structure that helps in the detection.
  • An event store where events are kept for further consumption by other agents -- this is a global (or bounded) state.
  • Reference data that is being used by event processing agents for enrichment, this data is not maintained by the event processing system, but can be considered as part of the event processing state, since the event processing results may depend upon values of this data.
  • Global variables (whether persisted or used on a shared memory) that are being used as a global state across event processing agents.
  • Context -- on which I have blogged several times, which may have temporal, spatial, segmentation and external state (which is indeed a global variable) dimensions.
All of them are in a way part of the system state, they have different roles, though.

BTW -- we have revised the building blocks that we are using in the EPIA book, by replacing event derivation (which is actually part of the event processing agent) with global state.

Among the types of states we mentioned before -- context is a first class citizen in the model, internal state is hidden inside the implementation and is not explicitly modelled (although some languages allow also to model the internal state explicitly) and global state contains all the rest (reference data, event store, global variables). More - Later.


Paul Vincent said...

Hi Opher - wondering how you relate "context" and "state" - is context the (sum of) state(s) of related entities (events and data)?

Opher Etzion said...

Hi Paul: Context is related to event, it is a way to group events together based on : time ( windows of various types are temporal contexts), space, segmentations, and entities' states.
Context is a state in the sense that it spans over time and there can be transitions in and out. One of the context dimensions can be state of some entity that is external to the event system



Anonymous said...

Hello Opher,

sometimes I act in a haptic way, so your answer seems to be quite philiosophical to me.

Can you explain me a little more in detail, which state you actually mean?

I can think of different states and contexts of an event.
- I agree on the point you describe the state of the event processing agent. This seems to be obvious to me
- I do not agree with the event store, this partial component actually just contains the events without having the event state to me. Their state is eather true or false for containing an event or not. So the state is not related to the event from my perspective.
- The reference data is from my point of view nothing else than just another event. I do not see any difference.
- The global variables are something like process states. I admit, that they exist, but the relation to a single event is not clear. I would rather say, that a global state is a subsumtion of the single event state within a certain process context
- Same on the last point in the previous section.

What do you think of the classification for state on single events and event processing components and context on process transporting parts?

Best regards,


Opher Etzion said...

Hi Christoph. My point is exactly that the word "state" when talking about event processing system is overloaded and can refer to multiple things. Typically an event is a transition, while states (whether transitioned-in by events or not) span over time, until they change (event conceptually never changes, although in practice it can be retracted). Thus, reference data is not an event, the value is there until changed, and typically it is not handled or changed by an event processing system, but by other system. Likewise global variable is not an event, it can be updated as a result of event happening, and it can matter in event processing, e.g. there is a predicate based on a global variable that participates in the processing of events, concrete examples: a certain agent applies only when red alert is in effect, or a derived event takes into consideration the current exchange rate between US Dollar and Israeli Shekel that is changed once a day (this is a reference data example).