Saturday, July 5, 2008

On events and Data




I am back at home for 2.5 days, in Monday morning I'll continue travelling the world - this time in Germany mostly for the DoReMoPat consortium meeting, which is related to the tutorial I've given in DEBS on event processing patterns. Today I would like to refer to one of the topics discussed in DEBS, as a result of the tutorial given by my friends Shailendra and Dieter from Oracle - the questions is what are the boundaries (if any) between database processing and event processing. I have already posted in the past a blog entry about the topic: is event processing a footnote for databases? and will not repeat what was written there, to illustrate the difference between event and data, I'll take the state machine example -- data is like states and events are like transition. Thus, a database reflects a snapshot that captures a single state in the universe. In event processing we process not the state, but the history of transitions, thus, the type of processing is slightly different (although as we noticed from the Oracle's presentation, there is now an attempt to take the pattern notion from EP and get it into SQL extension, IMHO - the result is somewhat complicated, since is not a natural hybrid), Anyway, one of the controversial point has been - whether sensor reading is an event or a data -- according to the state/transition test - a sensor reading is an event -- it transitions the database to a new state that includes the new transition. Some clever guy asked me the question - what happens if the sensor reading is the same as the previous one, nothing is change in the universe, so it is not a transition. Well - good question, but IMHO this is still a transition, first transitions may return the state machine to the same state (as seen in the state machine example above - the top state has a transition that maps back to itself). Why is it a transition ? -- first the timestamp is different, it is a new piece of information that exists (the combination of reading and timestamp) and has the potential also to change the state of the universe by mapping to another state. More - Later.

2 comments:

PatternStorm said...

Hi Opher,

an event is something that happened, clearly something that happened can be recorded/persisted as data, but then, what triggers the process of recording that something that happened?...Could it be that the time dimension plays an important role in the distinction of when something that happens should be considered an event and when data? Very roughly speaking, things that have happened and have been processed would be data while things that "are happening" or "have recently happened" and have not yet been processed would be events. For instance, following your state machine example, states in an state machine are just equivalence classes of the set of all the possible history of events that the state machine can process (i.e. each state is just an infinite set of histories of events) therefore, an event, after being processed becomes data in the sense that it is conceptually added to the end of the actual history of events already processed by the state machine and that "causes" the state machine to transition to the corresponding state, that is, to the state that contains the resulting history of events after having appended the last processed one. So, once processed, the event becomes part of the state...

Therefore...do events once processed become data?

Regards,
PatternStorm

Opher Etzion said...

Answer:

To continue with the line of thinking -- events = transitions and data = states, what is captured as data are the state, which is the result of application of transition to a previous state, the actual transition is not necessary kept as data (but transitions can also be recorded).

cheers,

Opher