Event Processing in Action - Last mile

This is the image from Amazon, which already reports on some selling of the EPIA (every day its sales ranking is changed). The book will be out in print sometimes in July (more towards the end of July I think). We are now in the last mile, the graphical editor works on redoing many of our figures in a more professional way. Yesterday there has been a new version of the book for the MEAP subscribers, so those of you who subscribed should have received a link to download it. This is the last version of MEAP, next version will be the final e-book. Like computerized systems, books can be improved forever, and I already have a (short) list of possible improvements for the next version (if there will be a next version), but at some point one needs to freeze and let the book be born. Some people have already written reviews on their Blogs:

Marco Sierio - recently and in the past and Hans Gilde in his first and second reviews. Looking in Google, I found that somebody has put it as a reference to the "Complex Event Processing" value in Wikipedia. When the book will be out there will probably be more reviews. The book also has a website hosted by EPTS that includes six implementations to the application example that accompanies the book (no login is needed to access it). More implementations are expected to be added. More -later.

On high availability in event processing applications

In the EPIA book (now due to be available in late July), we have a discussion about non-functional requirements of event processing applications. There is a variety of such requirements, and different applications require different set of requirements. One of the common requirements we seen nowadays is high availability. Recently an article describing building high availability in Websphere Business Events using IBM middleware was published in IBM developerWorks. This article was written by Chris Ahrendt, who has been until recently working in IBM software services, and currently and currently defines himself in LinkedIn as "Independent at consultant". Chris was one of the first IBM solution architects exposed to the event processing technologies early on, when we did proof of concepts with Amit, and was member of the evaluation committee we have done for the IBM Software Group Architecture Board, that was one of the milestones in the decision to get into this area. Chris' article briefly explains the concept of high availability and describe the specific solution. Enjoy!

On students' projects in the event processing course

The semester has come to its end, and the event processing course in the Technion had the last meeting today; the students still has to finish the projects, and they have some time to do it, but today they have presented a preview of the project, here is a brief summary. They were given a choice to do research oriented project or implementation oriented project, those who chose implementation oriented project got some directions on the size (quantities of consumers, producers, patterns, aggregations, filters etc...), and could chose both the application they want to work on (and they'll have to test it with simulated data) and the tool they use in order to implement.

There are five teams:

• Team no. 1 -- chose to work on a research problem of interval-oriented event processing and defined a collection of operators and patterns required to support interval oriented.

The rest of the teams chose implementation projects:

• Team no. 2 -- chose to work on "smart traffic control", simulating traffic lights, traffic jams, accidents and events like concerts etc... They decided to hard-code their application in Java, since they know Java well, and it is more fun doing it this way :-).
• Team no. 3 - chose to work on "smart home" and mainly concentrate on the security aspect of home automation, they are also Java fans, but chose to use Esper in conjunction with Java.
• Team no. 4 -- chose to work on control of people who are abusing chat rooms in various ways (like using dirty words), and managing complains. They have chose to work in Streambase, and done all implementation (so far) using the graphical tool of Streambase. We again see that some people like graphical programming and some people prefer old-style imperative programming.
• Team no. 5 -- chose to work on tracking bus driver performance, both for safety issues (speed), and accuracy (skipping bus stations). They have organized late, so have not started the implementation phase yet, but they said that they wish to do the project using Apama.

The projects are due in August, and then I'll write more, if I'll have any interesting insights.

On image and unfair disadvantages

This is the logo of the "Hebrew Reali School" - a famous Israeli private school, in which I have spent 8 years of my life, my wife and myself are both products of private schools and we sent our daughters also (at some point) to study in this school. Yesterday I spent all evening there, in a ceremony to celebrate my daughter Hadas graduation from junior high (9th grade), she will stay in the same school, but move to a different location for the "upper division". The coordinator of the 9th grade classes in his speech claimed that by learning in that school the children gain advantage over other children for their future, from my experience - this is definitely true. Some people may view it as unfair advantage, but that's life -- the public system provides a low common denominator, and if I can afford getting more than that, I am not ashamed of pursuing it. I admire people that I know who send their children to public schools because of ideology and values , but I don't share their attitude.

Somewhat unrelated --- I read in Marco Sierio's Blog that quoted an article in the Atlantic magazine about the unfairness of using high-frequency-trading, Marco asserts: When this is what the public reads about CEP, then I think we might have a minor image problem.

Some quick comments about Marco's assertion:

1. I did not find the phrase CEP in the article, it has talked about high-frequency-trading. If my memory does not mislead me, Marco's company has a nice event processing software-as-a-service offering, and its typical applications are not really high-frequency-trading. This is also true for what we do in IBM in this area (there are many applications in many industries), and other companies as well. Algorithmic trading applications were indeed some of the early adopters of event processing technology, and there are some vendors that are still centered around trading applications. However, one of the issues, that is indeed a misconception that exists among some people is the event processing = algorithmic trading, but this is only one of the applications, and there are many others.
2. Furthermore, there are some other applications that are indeed aimed at the benefit of humanity as a whole, and our event processing grand challenge as noted in the EC FET FLAGSHIP presentation (that we want to realize with the help of government funds) is certainly in that direction.
3. Now getting back to the unfair advantage, I cannot say that I have deep understanding of the trading issue, so unlike some people who have opinions about everything on earth, I tend not to express opinions about something I don't understand, so I'll leave the discussion to those who understand more and avoid expressing opinions here.
4. However, I'll write in general about using technology for providing unfair advantage. When I was young, I worked in the IT shop of the Israeli Air-Force, and saw how sophisticated informations systems have shifted the balance of power in organizations, gave one organization advantage over others, and eventually changed the organizational structure and missions. I noted how some people could take advantage of technology in order to advance the organizations they managed, it was fascinating lesson. Was it unfair to the others? certainly, one can think this way; but nothing stopped the others from doing similar things, those who made high priority in investing in technology benefited in more than one way, there is nothing special in that assertion. Event processing technology can certainly provide real-time observations or automated actions in many areas, and one cannot stop the progress of technology.
5. Again, I don't know if some traders abuse the system, this is a question to regulators, and I have no knowledge and opinion about it -- but it is well known the terrorists have used Email and SMS to coordinate among themselves. Every technology can be abused in some way.

Getting to the beginning of the posting -- I am sending my children to private schools to give them advantage in life; likewise the wise organizations take advantage of sophisticated technologies to get advantage in many areas -- and I still have no opinion about the high-frequency-trading, as I don't know much about it.