Moving to a new job - but actually staying in the same office

This is part of the Haifa Bay as seen from the IBM building (not from my office - I am in the back side!). On April 1st I am moving to a new job - without physically moving office (which means that I am still in IBM).

What role I am completing ? in the last three years I have been in a technical strategy role for the IBM Websphere product organization.

I have arrived to the product organization when people there did not really know much about "event processing" (well, IBM has done some forms of event processing in the past, but the people inside IBM did not think of that this way), I have worked on technical strategy, and business strategy, chairing and participating in workgroups, worked on business evaluation with the business side of IBM, working on comprehansive requirement survey in multiple industries, participating in many reviews, preparing answers for skeptic people etc.. I am leaving this position in a state that IBM sees event processing as part of its main stream middleware (as evident in the acquisition of Aptsoft, and some other activities), and as the next big thing.

Of course, the relationships between my presence and IBM's decision to jump on board the event processing ship may be just a coincidence...

Anyway - it seems that the next challange for me will be in shaping the next generation, while I still be somewhat involved in the work on the current generation.

One of my findings is that the "event revolution" has not occured yet, and I would like to go back to the drawing board, and do some work to address some fundemental challanges. In order to concentrate on this goal, I am re-joining to the IBM Research division and will focus on these research challanges (hopefully without many interruptions, each organization collects its own taxes on one's time, and IBM Research is no different ). What has not changed ? I am continuing to represent IBM in the EPTS and will honor all external commitments.

Stay tuned for call for membership in EPTS - coming this week.

My OMG talk

The picture is of Washington DC, as seen from the Arlington side. Two weeks ago in the OMG meeting in Arlington I gave a talk on EPTS and EP standards, the talk has been now posted on the OMG site, so you can download it from there. In the next few days we'll advertise the call for participation for EPTS, towards its official launch - stay tuned.

On Event Driven Marketing

Following my previous posting that talked about the effect of event processing on the life of some people, I came across another interesting article today that describes event-driven marketing, where event can be: "customer service call, a type of transaction, going through a spending level in a particular period, customer's birthday". This is different from traditional marketing, finding the right opportunity to issue personalized marketing is a capability enabled by processing events. More interesting applications - later.

Event Processing can change some people's life

After returning from a business trip, and a few days family vacation to celebrate the Purim holiday, I am retrurning to the Blog. One of my loyal readers, Eran Toch, has suggested the topic of this posting by sending me an article saying that the former NY governor, Mr. Spitzer, has started his way down of his high office, when caught by a "anti money laundering" program that looked for simple pattern of three events. Well - this is a classic event processing pattern, and it seems that it has changed the life of a few people (starting from Eliot Spitzer himself, and in a chain effect, many other people). This reminds me of a visit that a group of Eurpoean journalists visited the IBM Haifa Research Lab in Israel in 2003 or 2004, and I was asked to give them a talk about CEP (I am not sure even if we already used this name) - after giving them some example of use cases, all of a sudden one of the journalists had the association to Orwell's big brother, and after he mentioned it, there was some philosophical debate between some of the journalists to us, whether this technology has a potential to be a nightmare for human rights. My opinion is that many technologies can be abused, but can also used for good causes - criminals (and auditors) can use patterns to find flows in systems, and organizations can use similar patterns in order to trace the same criminals that are trying to take advantage of flows... No matter how used -- event patterns is very powerfull, and its impact on our life has not even scratched the surface of its potential.

Relativity and Event-oriented thinking

You may be familiar with the magnificent masterpiece of Escher called "Relativity". We see relativity everywhere -- on Saturday I've called home from my hotel in Burlington (since then I've moved south, and now I am in Hawthorne, NY) and told my daughter that I am sitting in the hotel waiting for the snow to end, in order to get out, and she said -- "how lucky you are to see snow", I have not felt lucky to see the snow, in fact, it kind of disrupted some pans I had, but in the world of a person that lives in Haifa, snow is a rare thing that occurs once every 20 years for a short time, so it is an attraction. At some point in my past I have been teaching a basic AI course, and the teaching assistant has taught the students Lisp. I have looked at the end of the semester on Lisp code produced by students - some got it right and wrote pure recursive program, and some wrote code that looked like Fortran thinking in Lisp.

The same relativity exist when dealing with event-based applications. Some people understand how to construct such applications, while other people think in other paradigms like request-respond, and are trying to do some blend here. It is perfectly OK to do an hybrid approach, but this should be a result of careful design, and not by chance. Bottom line - we need a methodology for constructing event-based applications and tools to support it - this will somewhat different kind of thinking... more about this topic - later,

The Babylon tower and event formats

Still in the USA, after the OMG meeting in the Washington DC area, I got to the Boston area, and now I am in Burlington, MA, visiting the (former) Aptsoft guys. When driving abroad I am renting a car with GPS (see above to determine which one), and it typically gets me to where I want - however, it is still not totally reliable, in the last day it confused me twice, one yesterday night to find the hotel, it told me to turn left, and meant in the next turn, not in the current turn, but it did not say so, thus, I found myself back on the I95, and had to turn around at the next exit, and try again. Today it sent me to some shopping center instead of the Aptsoft site, and after I ignored it and found it - the local people told me that the GPS maps have the numbers of the street in the wrong directions (starting from the other end) - well, sometimes the technology is fine, and the weakest link is the data it uses.

Talking about data, one of the topics that were discussed in the OMG meeting about standards is the topic of -- semantic/structural standards for events. I have used the term "Babylon tower" in one of the earliest postings in this Blog and meant that we have Babylon tower of languages - like the original tower who separated the languages. However, there is another Babylon tower that relates to event format - syntax and semantics, and here the problem is even harder, since there are multiple formats in multiple domains, nobody even made an inventory survey. One of the presenters said (and it is true in some cases) that 80 percent of the cost of building EP application is to set up the events from the sources and transform them to a processable format by composing adapters (hand coded, or by using transformation engines). This is some of the domains that need more investigation, and perhaps we need a meta-data standard here.

What can we learn from paradigms that succeeded ?

Today I have taken a few hours off my generally busy schedule, and visited the "National Air and Space Museum" in Washington D.C.; the globe shown above is showing stars as they were known in the 16th century (if my memory does not mislead me), the museum is very impressive (I have seen it 14 years ago, which was my last time in the Washington area, somehow it is not on my path). Anyway, I got here to attend (parts of) the OMG Technical Meeting. I was invited to give a talk in the MARS session about EPTS and EP related standards, as a prelude to a "complex event processing standards" workshop that will occur tomorrow (I'll be able to be only in the first half, need to catch a flight - so apologies to the other presenters, but I am quite familiar with the different players...). I have already written in the past on event processing standards more than once, so I'll not repeat myself -- if the organizers will post my presentation on a website, I'll advertise the URL. However, I would like to show one of the concluding slides I've presented (actually reused it, I've presented this slide before...)

This slide shows an example of a successful paradigm - relational database. In order to get there we still need a lot of brain power, we are not really there, we are now, as I have written before, in a similar situation to databases in the sixties, currently we can do standards on the interoperability issues, and on modeling - however, we still need to get to the "heart of the matter" -- a relatively simple theory of event processing -- similar to what relational theory did to databases. Working on a single frame of thinking will also add the brain power of the research community as done for relational database with the massive work on query optimization - in EP the optimization issues is not easier (I think they are more difficult, and cannot be done incrementally from other disciplines' optimizations). Are we there -- not really ! Can we be there --- I believe that the answer is positive... More than that, in order to get to the main stream of computing we'll need to do the equivalent to ALL items on the slide...

Some people who heard my talk were skeptical - which is fine, I am surrounded by skeptical people all my life, got used to it. They have good reasons to be skeptic, and I have a good reason to be optimistic -I have always been a best-case-scenario person, and know it does not work, but in the attempt one lands close enough... more - later.