On the production phase of the EPIA book

I have co-edited some "collection of articles" books that are common in the research community, there is certainly some logistics associated with it, but relative to producing a "mass market" book it is a children's game comparing with the production of the EPIA book that I have been writing together with Peter Niblett. The publisher of our EPIA - Manning, has various ways to ensure quality. Manning is a publisher focusing on computing related book, thus it views book development as a software development project (and indeed many of its books are code-driven). While the book was initiated by Manning and not by us, they have sent the outline we sent to bunch of reviewers, all of them people with deep knowledge in event processing. Some of the advices we got were very useful. Since we wrote in the outline that we are going to base the book around a single example, the advice of one of the reviewers was that we'll use an example that everybody can understand, the reviewer added that if we would chose an example from the financial services domain (as many of the EP papers are doing), we might create a communication obstacle with some of the readers, who will not be familiar with the terms. We followed this advise and created the "Fast Flower Delivery" (FFD) that already received multiple implementations in many languages. For each 1/3 of the book, we had a milestone in which they sent the book to a bunch of reviewers (in one of them there was a response from 14 reviewers). The reviewers were mix - some people who have deep knowledge in event processing, and some who don't know what it is, both types of reviewers sent various types of comments, some of them resulted in restructuring of the book (originally we planned 15 chapters and 3 appendices, we ended up with 12 chapters and 2 appendices, but with around 100 pages more than we originally planned)- This was just the development process, and it last for 15 months. A few weeks ago we have started the production phase -- it has a workflow (supported by electronic content management system): Another technical review by a "technical proofreader" - here we looked at PhD student in this area, and thus addressed faculty members active in this area; the person who did this job is Samujjwal Bhandari, a PhD student in Texas tech university, he has read all chapters, made comments and caught various cases of inconsistencies among the different chapters, then it moves to the copy editor who makes editorial modifications, and returned to us with many comments and questions that we had to answer, after doing this round, each chapter is going to the proofreader, who is doing another editorial pass, and then comes back to us with questions and comments, so we are doing another pass on all chapters. After that it goes to the typesetting. In the background there is a graphical work to redo our amateurish figures in a professional way, and then hopefully the book will be ready. Currently we passed the phases of the copy-editing, and now working with the proofreader. There are a lot of editorial rules, and style issues that have to be dealt with (Peter is much better than me in noticing the small details), and in fact -- amount of work is much higher than I anticipated, it consumes much of my free time for over a year now, as we both are doing it in addition to our daily work -- but I hope the result will justify the investment. More about the book - later.

On computer science research publication

I have read one of the recent issues of the Communication of the ACM - as ACM member I am getting the hard copy of this magazine and from time to time I am browsing through it. One of the articles that caught my eye is an editorial article by the editor in chief, Moshe Vardi. It summarized some opinions about publications in computer science, it turns out that most disciplines make journals as the primary vehicle of publications, and view conferences more as a social gathering where most submitted papers are being accepted, while in the computer science area, conferences are a very dominant publication vehicle, and as such, the "good" conferences are very competitive, in fact, people are writing in their CV, what is the acceptance rate of each conference they published in. Vardi's article shows support in opinions that computer science has to mature, and behave like all other disciplines. This observation is true, in my previous incarnation, as faculty member in an inter-disciplinary school at the Technion, I quickly discovered that other disciplines don't really understand the term "refereed conferences", and that conference publication "does not count". One of the wrong (in my view) academic metrics is to count papers, but this is not the topic I am discussing now. From the industry research perspective, the current state of conference-centric, is very comfortable. Since (unlike our academic colleagues) writing papers is a secondary occupation, then it is comfortable to write relatively short papers based on the conferences's page restriction, it is comfortable to tell management that you have a deadline for publication then to dedicate time for undetermined deadline that always gets lower priority to other things that have deadline, and it is more comfortable for management to let the reviewers do the filtering of who should travel to conferences. Despite these facts, I think that getting computer science to act like a mature discipline is the right way to go forward. The journals and conferences will need to be adjusted. Academic journals will also are also going through a phase of coping with the Internet era, and various business models for journals are emerging. The event processing perspective of it is that we should consider to found an event processing journal, when we feel that we have enough quality content for it. The numbers of submissions to DEBS are indication that critical mass might exist, I guess that this will be discussed in some of the coming meetings (like the Dagstuhl seminar),

On virtual vs. real in event processing

As a follow on to my last posting on virtualization of event processing, one of the frequently asked questions, and maybe somewhat open question can we get really to platform independent models that can be directly compiled to implementation of our choice in fully automatic way. This is one of the major topics I am investigating recently. Two issues are involved: Is it possible to automatically compile to a specific platform that may be semantically incompatible with the platform independent model, the second issue is whether we can create an efficient implementation without taking more platform specific considerations. The answer lies within the compilation process from the platform independent model into the specific platform. This was not really done in event processing yet, but problems of the same magnitude have been handled in other areas, and we intend to leverage the experience of people who have done it in order to see how far this can go. As a related issue, Jim Odell continued his series on event processing and agent technology, hosted in the TIBCO CEP BLOG .

Since the platform independent model we are using has "event processing agents" as its major building block, agent oriented implementation is the most natural one, and the compilation to such a model will be straightforward, however, the platform independent model should also be compilable to centralized implementations, where the agents are all hidden inside a single run-time artifact. My guess is that the shift to agent oriented implementations will happen gradually, I'll talk about it when I'll present my view on "event processing - seven years from now" in the OMG event processing consortium meeting next month. I'll write more in the future on the compilation from model to implementation as we'll get progress in that project,

On the virtualization of event processing functions

There is some discussion about scale-up and scale-out for measures of systems scalability, as indicated by a recent Blog of Brenda Michelson, I would like to refer to the programming model aspects of it. Parallel computing becomes more and more a means of scalability due to hardware development and barriers in the scalability of a single processor that stem from energy consumption issues. In event processing both parallel and distributed computing will play important role, as we a large, and geographically distributed event processing networks.

The main issue in terms of programming model is that manually programming a combination of parallel programming and distributed programming is very difficult, since many considerations are playing here. The solution relies on the notion of virtualization. The event processing applications should be programmed in a conceptual level, providing both the application logic and flow, but also policies that define nonfunctional requirements, since different applications may have different important metrics. Then, given a certain distributed configuration that may also consist of multi-core machines, the conceptual model should be directly compiled into an efficient implementation based on the objectives set by the policies. This is not easy, but was already done on limited domains. The challenge is to make it work for multiple platforms. This is part of the grand challenge of "event processing anywhere" that I'll describe in more length in subsequent posts. Achieving both scale-up and scale-out in event processing require intelligence in the automatic creation of implementation, and ability to fully virtualize all functional and non-functional requirements. More - later.

On the holocaust and the use of negative thinking

It is Holocaust memorial day here, a year ago I have written in this Blog a story about my father and King Alfred. This story brought me a lot of reactions. This year, the holocaust memorial day for me is in the shadow of my recent research about the family history, which I have neglected to do until now. The picture above is the famous gate to the Auschwitz concentration camp. I never had uncles or first cousins from my father side. Unlike myself, my father has not been an only son, he had seven brothers and sisters, five of them ended their lives in a single day in this place, the other two ended their lives even before getting to Auschwitz in Ghetto Lodz. Until two weeks ago I did not know most of their names, ,my father rarely talked about his family. My family history research revealed me all the names. The Nazis were very thorough and documents everything, today one can find a lot of information on the Web, though some effort is needed. My father was the second of the eight children and the only survivor. His elder brother Yehuda, died in the Ghetto in the age of 19, another of his brother, Chanoch, died in the Ghetto at the ages of 12. In that day in Auschwitz, my aunts and uncles whom I never met, were poisoned in the Gas chambers, like the people in this picture who march to their death.

The names are: Chana who was 18 years old, Eliezer - 12, Roza - 11, Rivka -10, and Miriam - 8 years old.

I would like to take this opportunity and connect the holocaust to another thread of off-topic posts that gained a lot of reactions, the one that talked on positive and negative thinking. There are people with negative thinking who just waste their energy on looking at the world in a negative light, these people are often annoying, but the more dangerous people are those who use negative thinking in order to get support for their agenda. It is always easy to unite people behind negative message, behind common threats, whether existing or imaginary. People are getting support to their political agenda by turning people against groups of people, the negative sentiment is unfortunately very effective way to get people's support. This is what enabled the extermination of millions of people, in what is known as the holocaust. This is also a warning for the future.

OMG event processing symposium and other event processing conferences

I was recently invited to give a talk in the OMG event processing symposium that will take place in Washington DC (actually, Arlington VA) in May 24-25, 2010. I will give a talk about -- event processing seven years from now --- talking about where the event processing area goes in the future, this will be a week following the Dagstuhl seminar on event processing, and some of the conclusions may be reported there. The OMG new event processing consortium that will be kicking off in that event is another indication to the importance that the IT community places on the event processing, and the visibility it gains. This is a customers' oriented conference, whose aim is to educate the general community about event processing.

We also announced that EPTS will be doing its annual conference in November, co-located with the Gartner Enterprise Architecture Foundations Seminar, in Los Angeles. In this conference EPTS (possibly in collaboration with OMG) will grant two kind of awards: event processing innovative foundation awards (for research work) and event processing innovative application award. Stay tuned for call for nominations soon.

More on family trees

Still holiday week, and I am spending some of it on investigating my family history, got to over 200 leaves in the family tree, and keep growing, on one branch of the tree I already got to the 18th century. This is a picture of a second cousin of my father, her name is Pola, and she was perished in the Holocaust in the age of 23, found her picture, since her father who survived the war, documented her, nobody can mistake her to be part of our family, she has striking resemblance to some of the people I know. This is an interesting process, and I am renewing connections with relatives that I've not talked with in years. Back to "event processing thinking" after the holiday in Tuesday.