Thursday, September 11, 2008
Wednesday, September 10, 2008
Today, our entire family has travelled to the "instruction basis" of the Israeli Navy, where my (second) daughter had the ceremony of finishing her basic training, above is the logo of the Israeli Navy, the Navy is the only branch of the army who has ceremonial white uniforms, so it was all white day...
This is also, somehow, the time in the year where many things related to event processing happen. Yesterday, IBM (the company who pays my salary) has done what was known internally as the "events event" in Boston for analysts and press, some report on it exist in the media, I think that David Berlind's report in "information week" is the most thorough one and includes video and slides. I have also seen several announcement by other vendors.
Next week there will be the two back-to-back events, the Gartner EPS - second of its kind, follows by the EPTS event processing symposium - which is fourth of its kind, but first one after the formal EPTS launch. The first one to hear analysts reports and some other talks geared to customers, the second - highly interactive, discussion oriented meeting geared to the EP community. This is primarily for EPTS members, but we have invited some guests. The EPTS meeting will be video-taped by CITT, and will be posted on the Web, so additional people willl have access to the material.
I'll blog more on these events next week. Looking forward to see the active those active in the community and also some new faces...
Monday, September 8, 2008
The scenario (translated to my language - without the "streams") -- Events are reported about cars that move through some segment of the road; each event consists of
There are also simultaneous events, i.e. several events that happen in the same time unit (what ever the time granularity is). The inputs are events of this type, the output is - for each event, generate a derived event that include the original attributes of the events and the average speed of cars in the same time unit. If you want to see the types of problems that the SQL implementators see in this simple example, read the streamsql paper. Instead of discussing SQL, I would like to show an alternative way to think about the same problem.
The slide below shows an alternative way to think about this problem - this is a very simple EPN (Event Processing Network) which has two functional agents, one producer (e.g. an event emitter that create events from video stream produced by a camera that looks at the road) and one consumer (whoever wants to see the output events)..
The two agents work under the same temporal context (it can be spatio-temporal if we also want to group by road segment) - in this case, a temporal context is opened and closed every beginning and end of 1 time unit.
- The raw event is called "car position event" and it goes to both agents.
- The first agent is an aggregator which calculates (incrementally) the average, since it is bounded to the context, the average is of events from the same time unit, at the end of the time unit it produces a single event "speed-average-event" with the structure
- The second agent is a "pattern detector" which takes two input events - the "car position event" again, and the derived event "speed-average-event"; the pattern that need to be identified is AND, and the "speed-average-event" for that agent has a consumption policy of "reuse" (which means that if an event can be used for multiple patterns). The agent produces a derived event - for each AND pattern that consists of the "output-event" whose structure is:
This EPN does not involve "streams" - the thinking is "event oriented" and it attempts to provide natural thinking about event processing functionality.
1. This is rather simple example, can also be solved by putting the average speed event on a global state (or event store/database) and then enrich it back - but the event-oriented is closer to the spirit of the original example which work on streams.
2. Aggregator and pattern detector are type of agents, there are some (not many) more types. Typically, an event processing network consist of multiple types of agents.
3. "Pattern Storm" claims that stream SQL ignore causality. One can view the relation between input events and output events of the same agent as a causality relation (he is using another scenario from the paper), and this can be set while defining the EPN.
One general comment (not related to this posting) - to "anonymous" - I'll gladly answer your question if you'll send it back and identify yourself. I don't publish anonymous comments.
I can post the solution to the rest of the examples in the stream SQL paper if anybody is interested...