Preparing for unexcepected events - pickpocket

Today I have experienced an unexpected event - I have been most likely pick pocketed while unloading to the car bags from shopping at the supermarket. When I noticed it - it was too late, the wallet was gone. It will take me some time and effort to reconstruct its content - had to cancel credit cards, get substitute for driving license, get new identity card, and some other less important cards as well. Event processing can help in this case -- first locating chip attached to driving license, credit cards etc - can help locate them (for the privacy fans - this will also let some big brother trace you), some automatic procedure to replace everything that has been there, without bothering me - would also been helpful. This bring the question of cost-effectiveness - the investment in handling relatively rare events which have a noticeable overhead vs. resolution when they do happen - this is a utility function which weights the cost, other damages, and the damage from the event that happens - an interesting investigation area - which also has practical impact on applying event processing in business situations -- which events should be traced and handled, and which not.

And since the tone of this blog is personal this time, I would like also to write a few lines about a good friend and colleague - Shlomit Zak - who unfortunately passed away recently in relatively young age. Many of the foundations of the work I have done over the years came from my joint work with Shlomit in the Israel Air-Force, many years ago, Shlomit has been one of the brightest persons I know, who, unfortunately did not come close to fulfill her potential; besides her professional abilities she has has an artistic soul, was gifted piano player, and keen sense of humor, and was famous by writing achrosticons. My last Email from her was two months ago when she sent me an Email "happy new year" card with achrosticon - and I have replied : "happy new year to you too - the achrosticon is nice, as usual; rhyming is mostly reasonable", and got the answer "and you - as usual, cannot free yourself from the need to provide grades" (well in Hebrew it sound better...).

The MARK on the BENCH - and the mythical event per second

Recent news item from BEA is talking about a benchmark and cites some EPS (Event Per Second) figures. Unlike some vendors that just cite numbers, there is also a white paper describing the benchmark. I don't wish to refer to the BEA benchmark specifically, but to share some insights about benchmarks in general. Benchmarks have a positive side, in which they are enable either to compare different products based on the same criteria, or to evaluate some properties of a product, even when not comparing it to others. Currently there is no "standard" benchmark in the event processing area, thus, vendors are inventing their own benchmark, carefully designed to expose much of the strengths, and none of the weaknesses of their products, and create benchmarks that may be non reproducible in other environments, or with some change in the application. Thus, to make any significant comparison between different products, standard benchmarks need to be constructed. Standard benchmarks, by themselves, may be double-edge sword, since we have benchmark-driven industry, vendors will invest a lot of resources into optimizing for the standard benchmark, however - this may not help a specific application, since its requirements may be far enough from the benchmark. Event Processing is heterogeneous area, which means that a single benchmark will not be sufficient - we need a collection of benchmarks, and each customer will have to chose the one or more benchmarks that are closer to its requirements. The standard benchmark should come from a vendor-neutral organization. I know of some academic work in this area, but more needed to be done.

And a word of caution - all the benchmarks refer to performance characteristics such as latency and throughput. But as noted in a previous post on the mythical event per second, I doubt if these are the main decision criteria in most applications - thus benchmarks should refer to other dimensions (functions, consumability, other non functional requirements), while, there are certainly cases that the high performance characteristics are critical, in general, I think this is over-hyped a bit. more - later.

On more or less important issues - the case of event processing languages

There are things important in life - and things that are less important.

In the picture you can see my four daughters ride two elephants in our family vacation in Thailand earlier this year, for me they are obviously very important, while other things may be less important in life. I am returning to the issue of event processing languages, since I realize that too much are being spent over the community on comparison of what a certain language syntax can or cannot do - I admit that I have also contributed my share to this discussion.

Now I am raising the question --- is it an important topic to deal with? yes - I realize that some vendors would like to make a language as a differentiator (I am not sure if anybody succeeded in this task), it is also true that some people in the development community have programming style preference, but since I see the trend of getting the patterns/rules/queries being constructed by semi-technical people, who will not be able to use many of the languages anyway, and will need higher level abstractions anyway.

Therefore, I think that the more important thing to understand are - requirements and semantic features of event processing - so I intend, at least for a while, to stay away from the programming style and concentrate more on these two issues.

more - later.

Context and Situation - are they synonyms?

I have spent the last couple of days in Eilat a resort town near the Red Sea, the Las Vegas of Israel, if one counts hotels, and an interesting combination of dessert and beaches in a small proximity, in the picture you can see the hotel in which we stayed. Anyway - back to context. A comment that received to the previous posting made me realize that there is a need to explain what is the difference between - "context" and "situation", both of them are semantic terms. In a way - it is similar (but not identical) to the difference between "state" and "transition". "situation" is something that happens, and that has some meaning in the consumer's terms, and it is the event consumed by the consumer(and as a result may trigger an action), in syntactic terms, situation is an event, and may be either raw or derived event (I'll not get into complex / composite events - it is sufficient to say it can be any kind of event). Context is a state -- a context can be created by the occurrence of event and destroyed by the occurrence of another event. Let's take a simple context: "during red alert". There is an event that declares the "red alert" state, and then there is an event that declares any other color, and it is considered to be "converse event" to the starting event, thus, it end the context. While the declaration of "red alert" is an event and can also be a situation, the fact that another event happened "during red alert" is not an event (semantically). Context may also be spatial - for fleet management application - a context may be a geographical area - and the events are that a vehicle enters and exits a context, but the fact that a "vehicle is within the area" is not an event.

Thus "context" has a distinct semantic existence. As far as implementation goes - there are various relations between events and contexts. After writing my first blog about context - somebody attracted my attentions that analysts have been recently talking about "context-aware delivery architecture" - I'll refer to this four letter acronym (are we out of all combinations of three letter acronyms?) in one of my next blog.

The notion of context and its role in event processing

I have copied this pyramid from an academic project site from Monash University in Australia that explores the notion of context in pervasive computing
It represents quite well the location of context - between the raw events ("sensory originated data" in this picture), and the derived events ("situation" in this picture).

The rationale behind this term is that under different circumstances, we identify different situations from the same event or combination of events. This can be temporal : "during working hours" we detect different situations than "during off-working hours" or "as long as red alert is in effect, it can be spatial : "if it happens within the county limits", or it can be entity-driven such as "if it relates to platinum customers".

Is context a first level abstraction ? - not in SQL oriented languages, where the "where" is semantically overloaded, although there is a limited support to temporal contexts with the notion of "time window" (which is usually just a time offset). Some works of context support in rules exist, though not in the main-stream products in this area. However, the benefits of getting context to be first class citizen:

• It is a first-class semantic abstraction: people think in contexts.
• It provides computational efficiency. E.g. in event processing network - context can determine the partition of agents, each agent does "function within context" (i.e. detecting a "complex event processing pattern", thus events are routed only to the agents that meet their context requirement.

I'll continue to discuss contexts in later blogs.

Blog **2 - A Blog about this Blog - after the first 1000 readers

Today I have processed the event that the number of "absolute unique visitors" is 1004 at the time I am writing this lines, and this deserves some reflections about this Blog. To be exact I may miss some readers in the statistics, since I have started the Blog in August 28, and started measuring in September 11. It seems, surprisingly, that people are reading this Blog, my previous experience with writing was writing poetry as a teenager, and this was long ago...
Anyway, I have drawn some statistics: Out of the 1004 readers, 12 have comments - the one who sent the highest amount of comments by far is Harvey Reed.
There were also a few who commented in their own Blogs - with or without explicitly mentioning me. The Geographical distribution is also interesting - most of my visits are from USA - 878 visits out of total of 1,798. This is true also for the "absolute unique visitors" - 525, which is the majority. So I am writing mostly for the USA folks - however, other countries have also contributed visitors to the "absolutely unique" list - UK is second with 89, India 44 and Canada 43 come next, and then France with 24, Australia and Israel with 23 each. The total is 59 countries - including one country that I have not previously heard of: Saint Kitts and Navis
as well as some of our neighbours - Kuwait, Iran, Egypt, Cyprus and Turkey - and countries in
all continents. Around half of the readers have returned, and some have returned more than 100 times (!!!). About the content - While most has entered the "root" - the most popular direct entries were those who dealt somehow with business rules: the most recent one and the first in this series. I am not sure they have made me very popular among the business rule guys (???). Next were the blog entry with all the TLAs and the mythical event per second piece. The most popular traffic source has been direct traffic (around 20%), and out of the referral site - the biggest by far is the almighty Google, however 10% of the traffic to my Blog has been referred from Tim Bass's Blog , I think that I have mentioned before that Tim has urged me for months to write a blog, so Tim's Blog is in a way my "parent Blog". About the content - I did not find strong correlation between the type of content provided and the number of visitors in that day, so I'll continue to experiment with writing all kind of staff (macro and micro issues) and react to other community Blogs. There are plenty of topics that I have promised to come back to, but will continue doing it in a random fashion. So - thanks a lot for the 1007 readers (the number has grown since I started writing this Blog), and see you in the next real Blog (this one probably does not count)...

XTP, Distributed Event Processing, and Distruptive Technologies

Hello everybody. Another experience of novice blogger - I had to start moderating comments, since I got spam attack of comments - so comments will not be seen in real-time anymore - but keep sending them anyway :-)

I have heard couple of times in the last few months talks from Yefim Natis
from Gartner who promotes now the concept of XTP - "Extreme Transaction Processing"
Yefim claims that XTP is a disruptive technology, in the way that it will be a new wave in enterprise application servers, he mentions in particular two disruptive technologies: dynamic grid architectures and event-based programming models. My friend Alan Lundberg asks in a recent blog what is the difference between CEP and XTP -- the short answer is that XTP is a framework that should have CEP capabilities but some other capabilities such as: transaction management, highly distributed environment etc... It is reasonable to think that some CEP applications will run on XTP, and maybe for some it will be overkill, but this depends on the ability of XTP to scale down as well as scale up. Another friend, Tim Bass (the person who talked me into writing this blog), has also recent blog about distributed event processing in which he said that "CEP engines should operate as cooperating intelligent agents" - and this assertion is the essence of the "event-based programming model" for XTP. I'll drill down into it in later blogs, but just point out some of the principles:

• In the event-based programming model - events and their flow are the most basic primitive
• Events are flowing among agents - agent may be intelligent agents that are doing "smart" things, or just software agents doing predetermined computations.
• The wisdom of the system is both in the agents and in the network - the network itself is intelligent and getting decisions.

This is another view of the world than the current programming models that are based around objects (and recently services), and may act as a disruptive programming model towards them, since having event as a first class citizen changes things.... These are preliminary ideas, but I find this idea intriguing and fascinating, so I'll probably dig into it deeper.

More - Later.