EPTS virtual symposium - March 24, 2011

Mark your calanders -- EPTS will hold its next (sixth) event processing symposium as a virtual symposium.
The idea of the virtual symposium is that a large audience will be able to participate through webcast, the symposium will also be recorded, so you'll be able to view the symposium or any part of it,   
The symposium will be co-located with the OMG Technical Meeting in Arlington, VA, USA, some of the speakers will be there in persons and the others will participate from remote.


The agenda was published on the EPTS site.    


The symposium will consist of three general themes:

1. Follow-up to the Dagstuhl seminar.    The concluding document ("event processing manifesto") in its final version will be published around the end of February, and the different teams will present their conclusions.
2. Discussion of event processing related standards:  this has been one of the manifesto chapters and there will also be presentation of Jim Odell about ideas within OMG around standards.
3. Report on several EPTS working groups activities

The EPTS business meeting and awards will be held in conjunction with DEBS 2011 in July.   

On events and decisions

Recently there are a lot of traction around events and decisions, coupling events and business rules, talking about decision platforms and decision servers,  meta-modeling efforts and more.  Some blogs dealt with the relationships between business rules and event processing - such as: James Taylor, and recently Paul Vincent. 


It is interesting to try and classify all the relationships between events and rules,  this is a rather streightforward list, no big insights:

1.  An event trigerrs a notification to a person for manual decision.
2. A derived event that is generated by an event processing system using filtering, transformation, aggregation, pattern matching or any combination of the above;  the derived event triggers a notification to a person for manual decision.
3. The next phase in the evolution is that raw or derived event triggers an automated decision based on either predetermined rules  or triggering an optimization process that generates a decision.\
4. In some cases the event processing system is the one doing the decision, since the pattern matched is the essence of the decision (e.g. who won the bid? ) and the event processing system itself triggers an action which is the result of the decision.
5. The opposite direction is that decisions also emit events:  events can be emitted to track the decision process in case it is multi-state decisions (e.g. procurement approval),  to check consistency among decisions, or consequences of this decisions in light of other events, and to match decisions with their consequence events in order to evaluate the quality of the specific decisions.
6. Events can also trigger modifications in the decision process, e.g. tuning parameters in policies, change the rules' segmentations and more

There may be some more interactions,  but these seem to be the main ones. 

On quality evaluation of scientific publications

Yesterday, the entire Etzion tribe visited the Technion to attend the ceremony in which my daughter Anat received a plaque of being in the president's list.    Since I don't have yet pictures from this ceremony I am posting pictures of the Technion president, Professor Peretz Lavi, and of Anat doing some non-academic tasks while renovating her apartment.

Talking about academia, in a recent posting on  the complexevents forum, Rainer has put a reference to a European project that is aimed to take a new approach to scientific publications.   In the relatively early days of the Internet (somewhere in the 1990-ies) there has been a panel in one of the conference (I think it was VLDB in Zurich) about the future of scientific publications in the Internet era.   The observation is that now a person can just post papers on the web and make them available to people, instead of publication in scientific journals (or conference proceedings, a computer science anomaly - most disciplines don't view it as a real publication).   However, the peer-reviewed publication process also serves as "quality control", moreover, in the academic world,  the promotion metrics are largely depend on publications (how much? where? how many citations?)  lead to metrics like H-Index and G-Index.   Since our world is increasingly metrics oriented, everything is measured, satisfying the metrics becomes the goal, and the way to do it becomes means (instead of vice versa), this is, by the way, true for other areas - not just scientific publications, but now we discuss scientific publications.   The original proposal at the panel, as far as I remember, was that everybody will freely publish paper on the web, but will get certification by peer-reviewed institutes, whose business models is that the author pays for review and they pay reviewers, authors can apply to several levels of certifications (grade A certification paper, grade B certification paper etc...).    Actually this did not happen, the scientific journals are still alive, although most accesses are now through digital libraries and not through hard copies.  I used to have a lot of journal subscriptions to hard copy journals, and when the IBM Haifa Research Lab migrated to the current building 10 years ago, I thought it is a good time to throw away all journals (leaving some journal issues with my papers published in) and move to digital libraries subsciptions.    


Now, the European project mentioned above, tries to re-invent the publication model claiming the failure of the peer reviewed system, saying there is low correlation between the grade achieved in the peer review process by three researchers who happend to be selected as reviewers and the impact of the paper in reality, and replace it by the "wisdom of the crowd", everybody can comment and endorse the paper.


As a side comment, I never liked the metrics of number of citations (always reminds me the old metrics based  on counting   lines of code), this is certainly a metrics for visibility, but not necessarily a quality mertics,  let's assume that somebody quotes my paper saying -- "this paper attempted to approach the same issue, but its solution was based on faulty assumptions", this is counted as a citation the same as another quote saying -- "we are basing our approach upon the seminal work done in that paper", again counted as one citation.  Also when citation number metrics became popular in evaluation processes, it encouraged the development of clicques that cite each other,  all of these are not reflected in metrics.  


Since I am working in an industrial research institute and not in a university,  my interest is to evaluate the impact of a scientific paper on the world,  where citations is only one (and maybe in some cases a minor) factor.   Codd's paper on the relational databases was cited by 1844 other papers (according to the  Microsoft Academic site)  but it is obvious that this is a minor consideraiton in its impact on the world. 
I think that one of the challenges is to measure this impact, which oftern can be done only in retrospect. 


More thoughts about it - later.

In search of intention language for event processing: revisiting the decision model

I have written recently about the need to establish intention language that will serve as higher level abstractions for event processing.  Last week, I had almost a day without laptop, when my laptop went to the system guys for re-installation of Windows (and thus all the software), due to some repeating failures.  I used one hour for my daily walk, some of the time to bother other people in the team, and had some time left for reading, so I decided to return to the copy of the "decision model" book that I have in my office, something that I had in mind doing since the visit of Larry Goldberg in my lab.    The decision model provides a tabular way of looking at decisions, with various propositions that are intended to enforce consistency, and normalization forms inspired from the relational model.    This expresses decisions that are assertion based,  and are intedned to serve as a semantic layer abouve business rules.   The event processing area is somewhat richer, as decisions can be based on events but also on situations that are based on detection of event patterns, or on aggregation or composition of events.  The results of event processing are derived events that may either trigger an action or issue a notification (which is also a kind of action).   The interesting question is whether this relatively simple tabular model can enrich its semantics in order to express the intentions behind event processing functions.    Getting such a cannonic way may be quite appealing,  however I am not sure it is going to be easy -- one direction to consider in search of intention language for event processing -- more on this one later. 

Event processing in simulation mode

You are probably familiar with the Microsoft flight simulator, that is seen in the picture, and indeed simulators became very common both in games and in training - such as pilot training which for many years heavily use simulators.   


In a recent blog James Taylor has written about the difficulty to validate new strategies, since strategic decision is translated to multiple tactical actions and possibly large number of operational actions. 

One of the possibilities to validate that the operational actions are consistent with the strategic planning is the use of simulation, where external events are simulated, and an event processing system is being used as a simulated business activity monitoring that checks the operational action results.   This requires the ability to codify the strategic decisions as  a collection of KPI (key performance indicators), which is typically doable.
Building the simulated BAM is more challenging,  first it requires some enhancements on event processing technology (on which I'll write in a later post), then it require to codify the operational actions and their effect on KPIs into the system.    I view event processing systems with sophisticated capabilities as a kernel for simulation and validation systems, not only in a corporate strategy level, but also in larger societies, simulating social processes,  or the impact of government policies.    One of the directions to pursue further - more about this direction later. 

On interfaces and event processing

Interfaces is one of the main next frontiers for event processing systems.  In the picture you can see an illustration of brain-computer interface which is the most advanced interface that is mentioned in a recent human-computer interface survey,  this survey starts from command line, moves through mouse and keyboard and gets to gesture detection, and up to the mostly futuristic brain interface.  My daughter Daphna got for her recent birthday XBOX machine with Kinect, which is capable of voice and gesture recognition and provide advanced game experience.   


Getting to event processing, the human computer interface is crucial in order to raise the abstraction level and enable to extend the type of developers and users, currently the development interfaces are geared mostly towards programmers, and are in essence close in nature to the implementation abstractions, a separation between the implementation abstractions and the development abstractions is one of the major challenges, in general in software, but an important one in event processing, since realizing the event processing potential entails the ability to make application development accessible  to other communities such as the spreadsheet programming community (e.g. business analysts).   I have written in the past about this requirement, and will probably deal with it more in 2011. 


Talking about interfaces, there is another type of interfaces the application programming interfaces (API), this is important since one of the major utilization of event processing is as embedded capabilities inside applications/middleware/solutions.   Thus, APIs that are appropriate for multiple uses are gaining importance for the interoperability and  for making EP  easily embeddable.   API design has become an emerging area, a blog posting by Shanley Kane from Apigee (and API management provider) provides trends and predictions for APIs.  It is interesting to note that the first prediction is entitled "APIs go real-time big-time" (in the source all words are capitalized, but I have adopted Manning's writing style while writing the EPIA book which is against over-capitalizing).   This prediction talks about the popularity for APIs of push oriented pub/sub systems.   This is elementary event processing,  APIs for more sophisticated event processing is the next step.
Standard APIs for EP will enable plugging EP components that are doing aggregation, translation and pattern matching components as part of an "event-driven Web" (AKA event-processing fabric) that was declared as the grand challenge in the EP Dagstuhl seminar.  I'll write more about this grand challenge soon.


In any event, both human computer interfaces and application programming interfaces are major part of the work needed towards the next generation of event processing systems. 

My 2010

In 2010 I have posted less entries in this blog, relative to 2008 and 2009, 128 postings, roughly once every three days. Maybe I've been busier, and maybe I've been lazier - probably the combination of these two. 

The happiest day of the year was surprisingly the day in August when I've returned from a family vacation in western Canada and found two things that got by mail:   

12 copies of the book EPIA that fresh out of press (I have already gave 10 of them as a present, left 2 for myself - one at home and one in the office); the EPIA book was a major task for 18 months, so I was relieved to see it in print, at some points it fell like never-ending saga. 

 

The second item that arrived it the same day is a plaque designating the fact that I've received an IBM corporate award, which is the highest award that IBM gives,  the order of the events was somewhat funny: first I got the plaque in the mail, two weeks later I got a letter signed by IBM's CEO, which notifying me that I am receiving the award, two weeks later than the list of awards was publicly published, and then after six more weeks there was the award granting ceremony, giving me again the plaque I got in August.

Another notable event was the Dagstuhl seminar on event processing in May 2010.   We are now in the final phase of editing the end result of this team work.  Dagstuhl is a wonderful place, and we had a very good team there spending  5 days in dealing with the present and future of event processing.

I had some trips abroad - both business and pleasure.   Unlike most years I spent only four days in the USA in October, for the OMG financial market conference,  spent two and half weeks in a family vacation in Western Canada,   here is a picture with my daughter Daphna somewhere in the Canadian Rockies.

Another trip was to VLDB in Singapore, with a couple of days vacation in Hong-Kong on the way, and several days vacation in Singapore.

Here is a picture from Hong-Kong's wax museum,  where I am photographed with an old friend.

Here are two pictures from Singapore, one from a zoo that resides in a rain-forest in the northern part of the island, and the second in a spa where you can get your feet cleaned by fish

Another conference I have participated in was DEBS 2010 in Cambridge, UK.

 This year I have also started the work on a new project that deals with proactive computing, and will write more about in in 2011.    

Overall --  interesting year,  the leaves a lot of unfinished challenges for the future.