Monday, June 16, 2008

On the right event processing language


Recently, the discussion about the "right" event processing language has been resumed in couple of Blog entries. First Mark Tsimelzon, the energetic CTO of Coral8, claims that the most important property is that it is familiar and thus similarity to SQL is a benefit. Louis Lovas from Apama argues that familiar syntax is indeed important, but readability is even more important, and argues for imperative language - since most programming in the universe is done in Java / C style; from the comments it seems that David Luckham consistently thinks that Rapide is the right answer. This discussion reminds me of the famous story illustrated in the picture above where a collection of blind people are touching an elephant in different places, and consequently each of them describes the elephant differently. Different people have arrived to the Event Processing area from different backgrounds, and have in mind different type of usages and applications, they also may see different types of users in mind - e.g. verification engineers can work comfortably with temporal logic language, C/Java programmers may have some inclination towards script languages, business users need somewhat higher level language etc... In some applications the main natural abstraction is "stream" as a set of events in a certain time window, and most patterns are set oriented (like: aggregation, threshold, trend seeking etc...), while for other applications the main natural abstraction is "event", since events are processed individually, and the type of patterns deal with individual events (like: conjunction, disjunction, time-out etc..). The challenge with the maturing of the area is to build a language with all the right abstractions, but we need to understand them well first. A comment about the language syntax seems familiar --- yes, it has a benefit, however following this type of reasoning, we would still be in Assembly languages which was what programmers were most familiar with when I started my career as a programmer (once upon a time...), we are advancing since then to provide abstractions. SQL is by itself an abstraction over imperative languages that has started someday, I have still written imperative queries with loops in the pre-SQL era, likewise, spreadsheet programming (which started with Lotus 1-2-3) does not look familiar, but became the most pervasive programming style that exists today (for non-programmers). Thus, IMHO, abstractions that enable to think naturally about a certain type of functions is more important than familiar syntax.