Then they asked me to provide some impressions
We were briefed beforehand and were told that we'll not be able to use slides, just use voice, so I have given some insights and impressions in a way that was described later by people as a "stand-up performance" (well, I was sitting down), this is consistent with one of my old habits - Ridendo Dicere Verum, when we started working on event processing in 1998, we did not know that we are doing event processing, we did not actually used that name, we also did not realize that 10-12 years later it will be an established and well-recognized area. Getting new ideas in a big corporate are never easy, and I have written before about some best practices to navigate.
Getting a recognition is always fun - but as my motto in life is based on the poem "IF" by Kipling, I'll end with one citation from this immortal poem:
If you can meet with Triumph and Disaster
And treat those two impostors just the same;
This award has reflected past work, but the challenge for the present is to work on the future of this area, my recent talk in the OMG conference reflects some of the work towards the future.
IBM recently announced "Websphere Decision Server". This announcement states that this offering combines business event processing with business rules management system to accelerate decision making.
There are several ways in which event processing and business rules interact, some of them are: an event, derived by pattern detection, triggers a rule, which makes it event-driven decision; the other direction is also valid: an execution of rules brings into decision and this decision can be reported as an event and may influence other decisions. Triggering decisions is one of the major uses of event processing (of course it is not the only one, e.g. event processing can be used for diagnosis or information dissemination), and it is a component in the automated decision making process, but these components mainly exist within islands (EP, business rules, optimization software and other means of decision support systems). One of the areas we are working on (we had a short report on it within the "fast abstract" session of DEBS 2010, as we were not in a position yet to deliver a full paper) is a unified conceptual model of event processing and business rules, where both are generalized as decision agents. this is still in the research phases, and not part of the product, nevertheless this announced offering provides step forward in achieving such an integration. More synergies in the decision space are expected.
Yesterday I have started teaching again, this time in the Bloomfield building at the Technion, seen in this picture. Sometimes people ask me why do I continue teaching, it is probably not because I am bored, on the contrary, I am quite busy; it is probably not because it is a well-paid job, actually they pay peanuts.
I think that I continue to do it, since I like the interaction with students, and it is also means to evangelize "event processing" to the younger generation; it also provides exposure to graduate students to work with, as over the years I have supervised (or co-supervised) 6 PhD dissertations and 21 MSc thesis, if my count is right. It is also source for recruiting. Quite a lot of my colleagues in IBM have been my own students (which I supervised), or my teaching assistants, or students that took courses with me.
The way my courses are designed revolve among the following principles:
For historical reasons the course is called "database engineering" and always there is a student who expects to see classical database course, for these students I answer that today in any database conference, data stream management is one of the emerging areas, and I am giving a flavor of it (my approach is not database centric,thought), I also show students some analysts testimonies that event processing is an emerging markets in general. The second principle is deep dive -- the course is going more on depth then on breadth, the students should gain deep understanding in what event processing is. The last principle is retain over time, my own experience is that to retain over time students have to experience themselves, so they'll get implementation project, I have not yet decided what the projects will be, gave myself time until next week.
I am using the EPIA book as a text book, but this time will not focus on the FFD example.
I might create teaching materials for this course so that others will be able to reuse it.
I noticed that the number of my linked-in contacts has grown again to a nice number 888, my wife said that if I'll hit 999 she'll join linked-in and be my 1000 connection, so have some way to go.
Anyway - one interesting insight from my recent visit in the USA is the way to react to event, or why proactive behavior needs to be smart. I was driving in a road and there has been a sign notifying: traffic jam in a certain bridge, it is advised to use alternative roads, there was still around 30 minutes to drive to that bridge, and a couple of alternative roads.
My own calculation was that most of the traffic heading to that direction will use alternative roads, and that there is enough time for the traffic jam to be cleared, assuming that there will be very little additional traffic, this of course was a gamble, but since I was not in a hurry to catch a flight, I decided to take this gamble, ignored the advice and drove straight to the bridge with the traffic jam, and surprise...surprise... the bridge was totally clear without any delay, indeed the traffic jam was cleared and there has been very little traffic heading in that direction. Of course, if many other drivers would have taken the same strategy it would not have worked (like the prisoner's dilemma in game theory). The lesson it that notifying about event may not yield the best result, maybe alternative road became jammed. In this case the global optimization was to direct certain percentage of the traffic to each alternative route and leave certain percentage directed to the original route. If we are looking from the point of view of individual driver who does not care about the others, this driver still needs to take into account the other's reaction in order to determine strategy. I guess that these kind of considerations are also getting into trading decisions.
Back home, looking at some notes I have taken in the OMG EP meeting in NYC last week. Here are some of these notes:
- The host was Credit Suisse, so we had an opening talk of Eric Newcomer, who told us about Credit Suisse, and also told us that his title was changed from CTO to Chief Architect, since the problem is not evaluation of technology, but coping with the complexity of the systems.
- Roy Schulte (Gartner) gave the first keynote talk. He set the tone of other speakers as classifying event processing as analytics (which is the current hot buzzword), he said (this is a slight variation about his previous classification) that event processing have three levels: the simple event processing, which we always done, the BPM + state machine which have some event-driven functionality, and the analytics event processing - AKA CEP or ESP, i.e. providing aggregations and pattern matching. On the financial market front he said that while the original EP applications have been in the front office, it is now getting to the middle and back office in various application areas. He also sees that the future of inter-systems integrations will be in using events. On EP as part of analytics he also talked on the BI classification -- descriptive (BAM systems), predictive (pattern matching is a way to define predictive) and prescriptive (which is still mostly futuristic). There are also three ways to deliver event processing: Basic engine, EP platform (which includes tools for management, debug etc..) and packaged applications (this classification is not new).
- Thomas Sulzbacher, CEO of Starview, talked about his product. He started his talk by saying that he took a flight from SFO to NYC in order to arrive into the conference, and in order to execute the flight, there were 12M event-driven decisions required, most of them automatic.
- David Parker, from Sybase, An SAP company, as he repeatedly reminded us, also classified event processing as part of the analytics solutions of Sybase. The interesting insight he provided (from Sybase internal sources) is how they view the distribution of the EP market: 42% Finance Sector, 14% government, 11% services, 8% telecommunication 8% healthcare and life sciences, 6% production, 4% retail, 3% education, 2% energy and utilities. It is interesting to see if other vendors have different perspectives, especially what analysts have to say, since no analyst has produced such a detailed distribution so far.
- Colin Clark, whom I met first time, talked and demoed his cloud event processing solution, that does indexing, clustering. classification, summarization and anomaly detection. He came along with his business partners that made a draw among the participants that provided them business cards, and gave some gifts -- I think it was iPad.
These talks were all before lunch - after lunch I did not take notes anymore; there were two talks - one by John Bates, interesting as always, given us concentration of incidents in the financial markets area, among other things, and the last talk (before mine) was Matt Meinel from Informatica (came from the acquisition of 29West).
Hello from Philadelphia again. Today I spent the day in NYC (got by train and returned to Philly in the evening).
I have attended the OMG event processing CoP conference for financial markets that was held in Madison Avenue in NYC. I'll write about the conference soon -- but for now, just a short posting about my own presentation; I have given a keynote address (actually in the program it has been written as luminary keynote, not sure I am a luminary, but it seems that Brenda Michelson who organized the conference likes to use this term). Anyway -- I have given in the first time a public talk about our activities in the proactive event-driven computing area, talking also about the "first of a kind" program of IBM Research which enables to do research work in collaboration with end users. I got several insights from the audience about various applications that I have not thought about for proactive computing, and will follow-up with some of them -- this is one of the directions to which the event processing area will evolve, and it seems to be an exciting direction to pursue, more about it - later.
I have finished my PhD studies 20 years ago in Temple University in Philadelphia, and after many years I have visited my old school today, and gave a talk in the CIS department seminar. It was interesting to meet again those faculty members who are there, and some new ones. I was especially happy to meet my advisor, Giorgio Ingargiola, who is about to retire at the end of this year. Giorgio has been a great mentor on clear thinking, and a major influencer on my own development, here is Giorgio's current picture.
I have some follow-ups, especially on the software engineering of event-based systems which I may have found a collaborator. Giving a seminar I got some interesting questions, one of them about the gap between events in computerized systems vs. events in the physical world -- which can be noisy and imprecise. I gave detailed answer about various cases where current systems have semantic gaps (this topic is discussed in chapter 11 of the EPIA book). Have to get up early tomorrow to take the train to NYC, for the Event Processing capital markets meeting.
