On P not equal NP

Last week a researcher from HP Research Lab, by the name of Vinay Deolalikar, notified the universe that he succeeded to prove that P is not equal NP, which has been an open problem for many years, many people has set to work on his more than 100 pages proof, and some claimed on finding fundamental flows, as a result Deolalikar notified that he is fixing some parts and will re-post an updated version soon. Many people are skeptic, but we'll have to wait and see.

Some observations about it:

1. The current Web 2.0 media -- Blogs, forums, social networks, is the frontier in which this entire discussion is being made, in a way there is a public peer review that is typically in scientific publications is being handled behind the scenes and don't make public. This is an interesting phenomenon. We see public reviews of books on the Amazon site, but public peer review on unpublished scientific work is rare to go public; I guess that this is currently reserved to VIP papers.
2. The scientific who did this work is from an industrial research lab and not from the academia, somewhat surprising since the image is that industrial research is interested in system oriented research and not in theory oriented research -- well, the world is not black and white. While research in industry is not in its glory period these days, it still attracts high quality scientists who prefer the frame of the private sector on the academic one.
3. People generally assume that P is not equal NP, and thus for many of the NP problems which have a need to be resolved in reality, there are devised methods and heuristics to obtain good enough solutions for most realistic scenarios. A proof that this assumption is true will have a huge theoretical value, but not a huge practical value, since it will not provide any insight of how to improve the solutions. Of course, if somebody will prove that P = NP it will have a pragmatic value, if it will show how to solve all the NP problems in polynomial time.
4. While the fate of this proof has not determined yet, I wish luck to Deolalikar, I like daring people.

On specialized graduate degree programs

Recently I came across several specialized programs, one of them is the master degree in De Paul university entitled "Master of Science in Predictive Analytics" that is sponsored by IBM; the other is a planned transatlantic master degree in collaboration between three European universities and three Canadians universities to pursue a collaborative program around BPM, in which event processing is one of the components. There are people who are strong supporters of specialized degrees, and there are also some opponents. Somewhere in the 1990-ies, I have been a member in a committee of the Israeli Higher Education Council, whose role has been to evaluate a new department (and degree programs) in one of the Israeli universities called "communication systems engineering", at that time this program has been controversial, the supporters claimed that it is the best education to a much needed profession, while the opponents claimed that it will create engineers with two narrow focus, and in general an engineer need to be more broad minded. At the end we recommended to approve this program, and it still exists, however, in some of the more conservative universities, it could not have happened.

Nowadays we have some tension between depth and breadth, and I guess that this is more of person own inclination where does the person classifies him/herself on the breadth vs. depth axis. The answer is not black and white, some roles are better occupied by people with depth and deep expertise, some roles are better occupied by people with very broad professional education. Of course, there are people who succeed to have both. Personally I prefer persons that are very good at something over persons that are mediocre in everything, moreover, one of the common mistakes of enterprises in employees development is to try and strengthen areas in which the employee is weak, in many cases, it is more effective to strengthen areas in which the employee is already strong, to challenge the employee to achieve excellence (unless the weakness is in a fundamental area that is critical for the employee's success).

Anyway, each of these programs will have to find its focus to create expertise in order to succeed.

On family vacation in Western Canada

In the picture you can see a rain-forest within the Pacific Rim natural reserve in the western part of the Vancouver Island. This area is less known then other natural reserves in the Canadian Rockies, but IMHO it is the most beautiful one, and we spent several days there at the end of our trip. We started our trip in Lake Louise, in the Canadian rockies Banff reserve, and caught by sleet storm in the middle of walking uphill (we climbed down in heavy slit that turned into rain), but besides the first trip, the weather was relatively cooperative. We continued north to Jasper, where we also had one of the nicest tracks, the Malinge canyon. There is a parking lot up the canyon, and another parking lot down, so one would expect that there will be a shuttle between the two parking lots, as climbing back is not really fun, it seems that the Canadians don't think this way - luckily I found somebody leaving the lower parking lot who agreed to drive me to the upper parking lot, so I could bring the car and take the rest of the family, in turn I also took another person who waited there for the same purpose -- and idea for a start-up!

We spent one day in the Yoho park in the way to Banff, and then continued to the second part of the trip -- flying to Vancouver, going in the ferry to Victoria and driving to Pacific rim (300 KM from Victoria), and then back to Vancouver through the northern ferry. Most of the trip consists on walks in the nature - glaciers, lakes, mountains, and in the west rain-forests. We also sailed in the Pacific rim to an island with natural hot springs, and then in the way back sailed through the ocean to see whales (we saw some). Good trip, I've returned charged with energy (and intimidately got flu, so last couple of days have not been so nice - and I am still behind on Email answering, but am recovering now).

Next trip: Hong-Kong and Singapore (for VLDB) in September.

The book: Event Processing IN ACTION - is now out

I still need to write something about the vacation in Western Canada, but returning to the office today, I have received a package of copies of the EPIA book, that was just published. The project of writing this book (in my spare time) was quite demanding, and was twice longer than the original expectation. I have talked with some colleagues who wrote books for other publishers recently, and found out that relative to their experience, Manning has exceptional quality control procedures, with three reviews by readers during the book's development, and a multi-stage production process with a lot of iteration between the authors and various people on the production team -- technical proofreader, copy editor, proofreader and the production manager.

The acknowledgements section of the book also lists many people who helped in contributing ideas and review and the Manning team; I am also grateful to David Luckham who agreed to write the Forewords section. Last but not least -- working with a partner on such a project requires the ability to agree on many details, and a lot of interaction, many of them in evenings and weekends. My partner in writing this book, Peter Niblett, has complemented me since he came from a different perspective; Peter's drive for perfection has contributed considerably to the quality of the book, Peter is also a very pleasant person to work with.

I also noticed that Manning added a section explaining who is the person on the front cover (some people asked me).

Manning maintains an Authors forum that enables communicating with the authors, this forum helped us during the book's development process to get feedback, and a lot of the comments have been adopted (with acknowledgement to the appropriate person); this forum is being kept alive.

The book itself is printed in black and white, there is also a colored ebook version, available on the Manning's book webpage. The book also included a use-case that is known as FFD (Fast Flower Delivery), and has already several implementations in different languages, with more expected, this site will be kept as live site and is being hosted by EPTS. Interestingly, the FFD case has been used within the DEBS 2010 event processing architectures tutorial in order to demonstrate the architecture notions.

The book has served as basis for both academic and IBM internal courses, and I'll be able to share teaching material with anybody interested to use the book for that capacity.

Some other follow-up ideas is to provide comprehensive authoring tool for the model described in the book, work on automatic compilation from the book's model into various event processing languages and maybe to general programming languages as well -- these can all be nice students' projects, the model described in this book can also serve as a first iteration on standard in event processing application modeling. We'll see how much of the follow-up will be materialized.

Family vacation

Later today I am starting a family vacation -- this time in Western Canada, mostly in Alberta, but some in British Columbia. I'll be disconnected from the Internet during this vacation - back on the air in August 4.

DEBS 2010 - the rest of the conference

Cambridge is also a nice place to walk in besides the King's college, most days I found time to take walks around Cambridge, and in one of them crossed the Cam river. It is recommended to visit Cambridge, even if you don't visit the university.

DEBS last day and a half included talks of both the research and industry tracks on various topics. From our team - Ella Rabinovich has delivered two talks: one in the industry track summarizing the experience of Amit over the years (I have written some background for this paper recently), and the other in the research track on analyzing the behavior of event processing applications. We got a lot of interest and have follow-ups in possible research collaborations, I'll write in details about the behavior analysis work in the future.

In addition -- more keynote talks, and also a follow-up meeting to the Dagstuhl seminar.

Got back home -- travelling through London, and had some time to take a walk in Hyde Park.

DEBS 2010 has ended, it was great to meet many friends again, and now it is the time to start working on DEBS 2011.

DEBS 2010 -- first day of the main conference

The first day of the main conference yesterday ended in an organ recital in the King's College Chapel, seen in the picture, magic place - people keep talking about being in Hogwarts. The conference itself started with the first keynote by Mike Franklin who shared his experience both from his work in Berkeley and in his start-up Truviso. Mike's main motivation is to extend database technology to react in a continuous way in addition to the traditional batch way of analytics, and view batch as a special case of the continuous. The talk was a kind of "lite sell pitch", but had some good points, like the observation that a start-up can take a known technology and try to use it for new types of applications (which is what Turviso does), or devise a new technology and tries to attack existing applications better, but it is very difficult to do both at the same time (new technology and new applications). I think that the event processing area indeed tries to do both.

There were some other talks, mostly by graduate students. One of the interesting observations (not a new observation to me), that there is a lot of energy in this community to re-invent wheels, in different variations. I think that one of the good results in the database community of the relational model was that a large part of the research community took it as a basis and constructed the research on top of it, and people did not try to re-invent databases from scratch for every thesis. In event processing we are still not there, and IMHO the area will have more substantial results, if the research efforts will be more focused on advancing the state-of-the-art instead of re-inventing most and advancing a little bit (in the best case). Today we'll have follow-up meeting to the Dagstuhl seminar, and the research grand challenge we are discussing is aimed to that matter.

In the late afternoon there were a fast abstract session, where I gave a short (8 minutes, 4 slides) talk on the interactions between business rules and event processing (I'll write about this subject some other time), and then a session of demos and posters, with a couple of follow-ups for me.

More -later. The second day of the main conference is about to start.