Event processing in practice

My old friend Roy Schulte from Gartner, who has been the first analyst that published analyst reports about event processing, and one of the founding fathers of the event processing community and EPTS has published a "personal blog" on the complexevents site with the provocative title:  " does anybody care about event processing?".   Roy expresses his opinion (that I've heard from him several times before) about the event processing concept and market. Roy's short answer to the question he asks is YES, but many of those who are doing event processing are not aware that they are doing event processing.   He mentions two main issues:

• Terminology:  the term event is manifested in many different ways, while the community centered around the current products have narrow interpretation
• Market:  Most of the event processing is custom code, or built into vertical products, and not the general purpose event processing platforms, thus he had to scale down his prediction for revenue that count the event processing platform market.

Roy suggests that EPTS will decouple itself from the focus on the product oriented functionality and concentrate on event processing as technology in the broad sense.   

I concur with Roy's observations (we have discussed it last year when I visited him in the Gartner office);  I have written before about the build-buy trade-off,  based on chapter 1 in the event processing manifesto, a chapter that Roy was one of its authors.      I also noted in some of the feedback to my book that I got from developers who told me that the book helped them a lot in architectural thinking and clarity of concepts, but they also said that they'll continue to develop event processing applications in custom code.  

Roy's Blog has been written in the context of the discussions on EPTS and online magazine,  and I have expressed my opinion in the same spirit as Roy --  the online magazine (and EPTS) should appeal to anybody that are doing anything with any kind of event in any form.  This should capture all the "event processing in practice" in the universe.  More news about the online magazine -- soon (the editorial board is now being created, which contains both old members of the community and new blood of practitioners).

More on EPTS and an online magazine

One of my most read posts on this Blog this year (actually 2nd place so far), has been a report on the all members conference call of EPTS we held earlier this year.    The idea discussed was to create online magazine as the major activity of EPTS.  After some delay,  in the last few weeks we have advanced in discussions among those who expressed interest.  There has been a keen discussion (conference calls and Email exchange) with some supporters and some concerns.  

Here is a background, and reminder about the history and current state of EPTS.

EPTS was formally founded in June 2008; there were some pre-EPTS activities in the form of event processing symposia, work on glossary and Dagstuhl seminar since March 2006.  EPTS also helped shifting the DEBS conference to become the primary research conference of the event processing community.   

The work in EPTS have been organized around working groups, some of the working groups have created visible results: the glossary is widely cited, the tutorial provided by the language analysis workgroup has more than 9000 views and downloads, the reference architecture workgroup provided several tutorials.  Some other workgroups did not succeed to take-off or create visible deliverables.  EPTS also partnered in initiating the Dagstuhl seminar in 2010 that generated the "event processing manifesto". 

There are 98 registered members in EPTS, 43 organizational members, and 55 individual members, among them 29 people from academia, and the rest are either consultants, or individuals working for non-member organizations.  Some of the members were active in work-groups and participated in meetings, and some were passive.     

Part of the mission of EPTS has been to educate the industry, and reach out to large audience, this mission has not been fully achieved so far, and much of the activities have been more inward-oriented among the members, these activities also decreased with time.  

Discussions late last year and earlier this year brought the idea of trying to reach out to wider audience and have more impact on the industry.  Alan Lundberg from TIBCO did some analysis and raised the idea to concentrate around a website that will be attractive to larger audience.   

In recent weeks we discussed the feasibility of such website and the way to succeed. 

Here are some of my  thoughts following these discussions:

The target audience should include  anybody that deals with events in any form (commercial EP products, custom code, embedded event functionality within products/packaged application, event producing -- such as sensors, event consumption, event transportation and more), and for any purpose (event processing application, event-driven BPM,  event-based decision management, event-based analytics, operational intelligence, continuous BI, Internet of Things,  event-based robotics, autonomic computing, bio-medical applications and more).  In the research community it should also involve both the community whose research area is directed towards advancing the event processing technology, as well as tangent areas such as: operation research, BPM, data mining, robotics, bio-informatics, sensors,  social media and more

 A critical success factor is that the type will appeal to various audiences to generate enough traffic.  Professional management of the site that will be backed on commercial basis is the preferred option;  We are now checking with an experienced person who might take the site ownership and commercial management. .     

A role model for such a site might be the RFID Journal  which seems to be a professional and popular website. 

Some concerns were raised about the feasibility both in demand and supply of content.   We have decided to make a feasibility assessment of these two sides (assessing the demand by doing survey, and locating qualified people who will be willing to be area editors within this website).

 If we'll determine that the feasibility assessment will look positive -- I'll propose to the EPTS members actions to make it happen,  otherwise -- we are back in square one.   Our surveys will go out in a couple of weeks and I'll report the progress on that.   More - later. 

EPTS activities in 2012

In Wednesday we held an open conference call of EPTS members.  The call discussed EPTS activities for 2012.  While in the past EPTS concentrated in working groups that have created glossary, reference architecture, investigated use cases and languages, and some other stuff, the idea is that in 2012 EPTS will attempt to address larger audience by creating online magazine which will consist of:  news (including connecting to social media),  general material about event processing (starting point: FAQ taken from analysts in this area),  and industry specific content.   We are now looking for volunteers to set up all aspects: orchestrating the content ("editorial board"),  soliciting readers and topics they are interested ("reach out board"), and a coordinator for sponsorship and advertisements.    Anybody interested to participate - please let me know.   I'll post updates in this Blog

EPTS event processing glossary 2.0 is now available

The event processing area keeps evolving and there are more terms being used.  The EPTS glossary workgroup, co-chaired by Roy Schulte and David Luckham has published a new version of the EPTS glossary.  It  contains both old and new terms.    

On event processing related standards

One of the items in the EPTS charter talked about incubating standards related to event processing.   When EPTS was established, three years ago, the general opinion was that it was still premature to deal with standards.
Recently, there have been two developments related to standards:

1. The event processing manifesto work includes a chapter that surveyed related standards and recommended some action items.   It has been presented within the EPTS virtual symposium last month by Paul Vincent.
2. OMG has renewed its call to express interest in work towards a standard on Event meta-modeling and profile  (in essence:  event structure).

Today, a vote request was sent to all EPTS members to answer two questions:  whether EPTS should establish a work-group to view standards in a strategic way, and whether EPTS should support/play a role in the OMG standard.   


Standards have a potential to give a big push to the area (this has happened in other areas);  the grand challenge presented in the event processing manifesto, the "event fabric" in an internet scale requires the existence of standards as a prerequisite,  however there need to be a shorter term business motivation to make it fly.

Reminder: EPTS awards

 The submission time for the EPTS awards that were announced last year are getting closer. 
 The awards (in cooperation with the OMG event processing community of practice) will be given in two categories:   Innovative application awards  and Innovative principles award 
These awards will be given for innovative applications or research ideas in event processing.    This is an opportunity to get recognition for innovative work.     The submission instructions can be found in the two links above.   The competition is open to all (one does not have to be EPTS member to apply), and to both industry and academic people, as well as those work for software vendors in the event processing area.       

EPTS virtual symposium - March 24, 2011

Mark your calanders -- EPTS will hold its next (sixth) event processing symposium as a virtual symposium.
The idea of the virtual symposium is that a large audience will be able to participate through webcast, the symposium will also be recorded, so you'll be able to view the symposium or any part of it,   
The symposium will be co-located with the OMG Technical Meeting in Arlington, VA, USA, some of the speakers will be there in persons and the others will participate from remote.


The agenda was published on the EPTS site.    


The symposium will consist of three general themes:

1. Follow-up to the Dagstuhl seminar.    The concluding document ("event processing manifesto") in its final version will be published around the end of February, and the different teams will present their conclusions.
2. Discussion of event processing related standards:  this has been one of the manifesto chapters and there will also be presentation of Jim Odell about ideas within OMG around standards.
3. Report on several EPTS working groups activities

The EPTS business meeting and awards will be held in conjunction with DEBS 2011 in July.   

Announcing the EPTS awards

EPTS has announced on a competition to grant awards of two types:

The innovative application award, in cooperation with OMG Event Processing Community of Practice, will be awarded to an application that have used event processing in an innovative way.

The innovative principles award, will be awarded to a research contribution that had a significant impact on the state-of-the-art in event processing.

The competition is open to everybody. Deadline for nomination submissions is September 30, and the award will be given in November and will publicized in the media.

Full details about the award program can be found on the EPTS external website.

Next week in DEBS we'll hand out a flyer with the announcement and start communicating the award program

OMG event processing symposium and other event processing conferences

I was recently invited to give a talk in the OMG event processing symposium that will take place in Washington DC (actually, Arlington VA) in May 24-25, 2010. I will give a talk about -- event processing seven years from now --- talking about where the event processing area goes in the future, this will be a week following the Dagstuhl seminar on event processing, and some of the conclusions may be reported there. The OMG new event processing consortium that will be kicking off in that event is another indication to the importance that the IT community places on the event processing, and the visibility it gains. This is a customers' oriented conference, whose aim is to educate the general community about event processing.

We also announced that EPTS will be doing its annual conference in November, co-located with the Gartner Enterprise Architecture Foundations Seminar, in Los Angeles. In this conference EPTS (possibly in collaboration with OMG) will grant two kind of awards: event processing innovative foundation awards (for research work) and event processing innovative application award. Stay tuned for call for nominations soon.

Towards an event processing manifesto - take one

The word manifesto brings to many people an association with the Communist Manifesto by Marx and Engels, which is an example how manifesto can be used, reused and abused. The term manifesto has been adopted by the computing community, and we have seen variety of works entitled: manifesto, some of them are: object oriented database manifesto, the manifesto for agile software development, and the business rules manifesto - just to name a few examples.

In May 2010 I am co-chairing (together with Mani Chandy and Rainer von Ammon) a Dagstuhl seminar, whose task is to compose "event processing manifesto", which we strive to make it as influential effort. A Dagstuhl seminar is a five days retreat in an isolated place in which a selected group of people are convened to make deep discussions about a certain area in computer science, many of these seminars are geared towards the academic community, but it this particular one we strive to keep the balance and have half of the participants from industry (vendors, independent consultants, analysts, and some innovative users), the number of attendees in each seminar in limited, and there is a big competition on this resource (and long waiting list after approved, we have put the application to do it in 2008). Here is a picture of Schloss Dagstuhl (located in Saarland, Germany):

I guess that I'll write a lot about the Dagstuhl seminar, the event processing manifesto, and related stuff in the next few months, so this is the modest starting point.

The intention of the manifesto is simple (but not easy) -- define what event processing is, what is the scope of event processing, are there mandatory features in event processing, what are the various options and variations, what are the synergies with other disciplines, and have as an appendix a call for action for the community: what are the research challenges and what are the pragmatic challenges.

I'll start today with some of my own thoughts on the scope issue.

I view "event processing" as a name of a computing discipline like "data management", "image processing" and "information retrieval" - relatively wide disciplines that cover wide collection of applications. This is also true for event processing. Alas, we still have some misconceptions even on the scope, I have used before the metaphor of blind people touching various parts of an elephant and concluding that each of them has identified what an elephant is, so this time I'll use another animal - ant whose sight is limited to its immediate environment.

I am using ant, since one of the great Hebrew poets has written a poem saying (in free translation): "my world is narrow as the world of an ant". Some examples of it:

1. people who view event processing as aggregation of time-series events, this is definitely an event processing application, but not the only one;
2. other people are looking at detection of anomalies within event collections, where the anomalies types are not predefined as what event processing is all about, this is indeed another event processing application, but not the only one;
3. yet other people are looking at real-time matching of predefined patterns with streaming events, which is somewhat different from the two previous examples.

One might claim that there is no event processing discipline at all, but there are very distinct areas that process events in different ways for different purposes, probably using different technique; while this may be a possible conclusion from the Dagstuhl meeting, my experience shows that there are many applications that require blend of event processing functionalities, and that the space is not discrete, but has some continuity, thus event processing in the wide sense should deal with all of them (and some more) and the synergies between them. So this is a point for discussion.

While the amount of people that will be involved in the actual composition of the manifesto is restricted, we'll strive to disseminate the draft to a wide discussion, both within the EPTS community and other avenues. As a comment, the Dagstuhl seminar is not directly related to EPTS, many of the attendees are not EPTS members, and some are members of adjacent communities and not the event processing community, however I think that EPTS should have a role in follow-ups.

More on what's next for EPTS

This is the logo of the event processing symposium, the building in the picture is opposite the conference's site. Back home now, after the event processing symposium and also co-located meeting of the EASSy consortium that is submitting a proposal for the EU R&D program.

This is a good time also to summarize the output of the symposium on the question what's next to EPTS. Earlier today this summary was sent to the EPTS members.

The main agreed goal of the "next steps" is Accelerating the activities and impact of EPTS, in order to do it, we'll need more people to enter the circle of activities in EPTS, some of the participants in the symposium, who have not been active so far, expressed willingness to participate in activities, and we are, as always, calling on more people to join.

Here are the activities, classified according to the activity area.

1. Existing working groups:

1.1. Glossary
There has been a discussion that included a conference call.
It was agreed that all comments and proposals from other work groups about terms in the glossary should be posted on the members wiki during the next month, after that time the glossary team will consider all comments and propose a revised version.

1.2 Use Cases
The use cases work group will publish its questionnaire and solicit use cases, also need to think of motivation to provide these use cases (see proposal on awards).

1.3 Language Analysis
The language analysis work group was asked to take the languages dimensions to the next level, so languages can be classified based on criteria. Also it will collect patterns and devise a library of patterns.

1.4 Reference Architecture
The reference architecture work group will continue its work to publish the reference architecture(s) of event processing.

1.5 Interoperability
This workgroup has been delayed. During the meeting it was re-established and will start activities.

There has been a general agreement that the workgroups should accelerate their activities; it was proposed that deliverables will be linked to external commitments (e.g. presentation/tutorials in external conferences) so deadlines will be established.

2. New proposed workgroups

Four new workgroups have been proposed. Each one has assigned a leader to write charter that will be put to a vote by all members:

2.1. Awards workgroups.

Awards will increase visibility, provide motivation for people to report various things (e.g. use cases) and establish EPTS as a recognized authority. Types of possible awards:
Research innovation award – given to researcher, Application innovation award - given to customer.

2.2. ROI

It was proposed that EPTS will publish a document about ROI of event processing in general and in different domains and industries; this will be used as a source for educating customers in a manner that is independent of a specific product.

2.3 EPTS promotion

It is proposed to promote the EPTS brand and awareness to its activities using website, webinars, white papers, press releases, participating in various conferences and more. This workgroup will recommend and coordinate these promotion activities.

2.4. Collecting Datasets.

Researchers require datasets for various research activities. This workgroup will help collecting these data sets, and makes EPTS as source of data sets repository.

3. Other activities


3.1 Coordination among work groups.

The issue of coordination among workgroups has been raised; the steering committee will work with the work group leaders to determine best way to act.

3.2. Grand challenges

There has been initial discussion on grand challenges. This discussion will continue on the members' Wiki. The Dagstuhl seminar in May 2010 will deal with the grand challenges issue.

3.3. Reach out to adjacent communities:

EPTS will continue to trace and participate in joint activities with the BPM, IT event management and robotics communities.

3.4. Association with DEBS

It was agreed to offer the DEBS steering committee to include EPTS as "in cooperation with" on DEBS website.

5th Event Processing symposium - second day

Still in Trento, here is another view from Trento's center city. Today we have started the day with the second reach-out session, this time to the IT management (AKA network and system management) area. The second keynote, Kristian Stewart, from IBM (formerly Micromuse) gave a good survey about the area, followed by a panel. This is an interesting domain, and we'll continue to look at the synergy between different domains of event processing. Next I have provided the report of the language analysis workgroup. There has been some long discussion about standards and next items. Directives for the workgroup:

• Collection patterns is a good idea.
• Get the dimensions work to the next level of detail, so languages will be able to be profiled on capabilities matrix
• Some opinions were to layer the functions in different layers. There were also counter opinion, but we'll take the issue back to the workgroup.
• As for standard, most people thought that we cannot achieve language standard in the near future, and interoperability standard should be a better goal.
• Meta-language may be attempted, maybe as extension of other meta-languages.
  

After the lunch break, we heard the third keynote speaker, Sebastian Wrede, who talked about event based systems in robotics. It was interesting to see applications in somewhat different domain than we are used to. We also had a bunch of short research presentations. Last but not least, we had a session on grand challenges on which I'll write a separate posting. After that we have been driven to a castle outside Trento for a dinner. It is late --- so I'll stop here.

What's next for EPTS

Next week we plan to hold the event processing symposium, fifth of its kind, and second since the launch of EPTS. One of the discussion topic will be -- what's next for EPTS ? We had some starts, got some traction, and now we need to decide on the next steps. So we'll dedicate half a day for this issue, in the last day of the symposium. Meanwhile, I am packing to go for a short vacation in Italy, and celebrate the new year there on the shores of Lake Garda, before going to Trento. Will be off-line for a few days.

On event processing grand challenges

This is the logo of the "Grand Challenges SIG" of the UK Electronic design knowledge transfer network. In the 4th Event Processing Symposium last year, Arkady Godin from MITRE, proposed in the business meeting to launch a work group that will deal with grand challenges in event processing. The EPTS steering committee decided that such a workgroup may be premature, and it is better to complete some work in the other workgroups -- languages, architecture, use cases, interoprability, in order to have a better understanding of the grand challenges, however, this issue has neither forgotten nor forsaken. We determined that the best forum to discuss the grand challenges is the (second) Dagstuhl seminar on event processing, which I'll co-organized with Rainer von Ammon and Mani Chandy, we also asked to invite all people that we saw as the right set of people to participate in such an event (if you have not been invited and think you would like to contribute, please let me know, there is a waiting list for available slots if any, no promises though).

In the coming 5th event processing symposium in Trento, we shall hold the first brainstorming about grand challenges. Pending technical feasibility (still requires confirmation from local organizers) we plan to do this as a public session that all EPTS members will be able to use through audio conference call, EPTS members will get full details with call-in numbers when it will be finalized.

Grand challenges can be in multiple areas and also refer to multiple addressees. There are grand challenges that will require a community effort, and these challenges should be picked up by EPTS, some challenges are to advance the state-of-the-art, and this will be addressed to the research community, with some creative incentives. Some will be referred to the product vendors, and may be some to other adjacent communities.

Idea for such grand challenges are solicited from various sources:

• EPTS workgroups that already are active, each workgroup will be asked to contribute a challenge in its area
• The reach-out sessions for other communities in the symposium (the BPM community and the the IT management community) might also lead to some challenges
• The research community is always a source for such challenges
• Last but not least -- all EPTS members (and other interested people that wish to contribute, and thus are hopefully future EPTS members), specifically customers who has long-term vision about their systems, like our MITRE colleagues who started this discussion, and our analysts colleagues who have cross-vendor and cross-customer perspective.
  
  Anybody who wishes to contribute ideas to this session, please let me know by the end of this week.

I'll wrap-up this posting, by pointing out a Blog entry I have seen just now, written by Niels, who runs a consultancy company called "SQL Develop" as its name testifies deals with DB related issues. The upcoming Microsoft product "StreamInsight" has raised the interest in the database community, as Microsoft seems to take an SQL oriented approach to its product, and Niels is blogging about "CEP resources" he covers Blogs in this area, on my Blog he writes: THE blog about CEP, if you were to read only one blog about CEP, this is it! First, thanks to Niels for the endorsement, I don't view the Blogging area as a competition, as people who read Blog tend to read multiple Blogs, and it is also kind of a network of Blogs that sometimes react to each other. To be fair, also most Blogs in this area are marketing oriented Blogs, that they have somewhat different motivation from my Blog, and one cannot compare. Anyway, a good way to start the day.

On the criteria to evaluate event processing products

This is a map of Finland, the location of our family vacation for this year. The vacation is planned to start in Saturday, and I'll be disconnected from the cyberspace for 15 days. Working late at night to advance in the EPIA book we are writing.

It seems that this is the time of the year of analysts report, the community blogland was full of references to the Forrester wave report - Complex event processing platform, Q3 2009, dated August 4, 2009.

I will not comment about the grades that they gave the different products, the reason for that is that I decided that in my role as chair of EPTS, I see my role to work on the coop side of the coopetition, and leave the competition side of the coopetition to others. I think that the main competition is not between the different vendors (though they have point competitions), but against the barriers that the event processing area has to fulfill its potential and become a pervasive main-stream technology.

The Forrester reports starts the executive summary by saying: "Forrester evaluated nine complex event processing (CEP) platforms using 114 criteria".

Without getting into the long list of criteria (not part of the report itself, but I managed to look at it), I have some doubts about the ability to get a meaningful information to customers by weighing 114 criteria. There are two reasons, one is practical and one is methodological.

On the methodology side, the compensatory model of decision making advocates weighing of many criteria, however, experience shows that the actual decision making model is lexicographic, meaning ordering the criteria according to importance, and making the decisions according to the most important criteria. People may use a compensatory model of weighing a lot of criteria, if their organization require them to work this way, but this is done only as justification to decision that has already been made by the lexicographic model.

Let's move from decision making theory to the event processing universe. The event processing universe is diversified from both functional and non-functional requirements point of view. I really don't believe in a "one size fits all" in anything related to this area, and this goes also for set of evaluation criteria. Getting criteria that are good in variety of cases and weighing them together may not get a good solution to any particular case. The more practical approach is to set a collection of relatively small sets of important criteria, and also segment the space of application, and assign a set of criteria to each segment. Anybody that will manage to do it, will help customers more to make the best decision for a particular case. I hope that EPTS, through its use cases workgroup will be able to provide this segmentation, and this will be the starting point for analysts to come with the right criteria for each segment.

On the Gartner 2009 application architecture hype cycle

Here is a revised version of my Blog entry that relates to the Gartner Application architecture hype cycle report (Gartner Report ID number G00168300 from July 16,2009) , the revision was done at the request of Gartner who asked that I'll make exact citations in their report, and make clear distinction between what is quoted from the Gartner report, and my own remarks.

Here are a collection of citations from the report that are of interest from the Event Processing perspective:

1. "Event-driven architecture (EDA) is an architectural style in which a component (or several components) in a software system executes in response to receiving one or more event notifications". In the report EDA is positioned under the hype cycle phase "Climbing the slope of enlightenment" which according to Gartner's terminology is defined as " Focused experimentation and solid hard work by an increasingly diverse range of organizations lead to a true understanding of the technology's applicability, risks and benefits. Commercial off-the-shelf methodologies and tools ease the development process"
2. CEP is positioned under the hype cycle phase of "Technology Trigger" which according to Gartner's terminology is defined as "A breakthrough, public demonstration, product launch or other event generates significant press and industry interest", and is the phase that precedes the "peak of inflated expectations" phase.
3. For CEP: "market penetration is 1% to 5% of target audience"
4. CEP use is expected to grow at approximately 25% per year from 2009 to 2014, but the use of COTS CEP products is expected to grow more than 40% per year in this time frame
5. For CEP COTS products: " Most of these products are immature and incomplete"
6. "Most business analysts do not know how to identify business situations that could be addressed through CEP, and that is limiting the rate at which CEP use can expand. Most software engineers are not familiar with CEP development"
7. "The Event Processing Technical Society (EPTS) was launched in June 2008, and it is expected to facilitate the adoption of CEP".
   

Here are my own comments:

• Note that EDA and CEP are positioned in different phases of the hype cycle.
• The fact that the market penetration is low indicates that there is still a substantial growth potential, if we can overcome the adoption challenges
• The adoption challenges consist of product maturity and market awareness. We are now still in the first generation of products in this area and maturity is typically achieved in later generation. Awareness and understanding of value and positioning are indeed a challenge.
• EPTS indeed has been formed to facilitate the adoption of the event processing area. Both challenges mentioned here – advancing the state of the art to accelerate the next generations, and educate the general community about the value and positioning of event processing within the enterprise computing.
  

Call for active participation in the 5th EPTS event processing symposium - Trento, September 21-23, 2009

As we are now in the high gear on planning the 5th event processing symposium, here is a call for active participation. It will be phrased in Q&A format:


Question: What is the Event Processing Symposium ?

This is an annual meeting, intended primarily for EPTS members, but open to other invitees up to the capacity limitation. It features some presentations and a lot of panels and discussions in the areas that are geared towards the understanding and advancing the state of the art and state of the practice in event processing. The previous symposium's program can be found on the ep-ts site.

Question: Where will the symposium take place?

The symposium will take place in Trento, Italy. The four previous meetings have been held in the USA, and since there is a big event processing community outside the USA, we thought of having the next one in Europe and call the large EP community in Europe to participate.
Trento university offered to host the meeting, they also host the symposium's website which currently includes information on the venue, and a registration form, and will also contain the program in the future.

Question: Who can participate ?

As said, the meeting is intended for EPTS members, however, other interested persons can apply and ask to participate. Based on the past experience, there will be some slots for non EPTS members. Participation is by invitation only, and requests for invitations can be done using the symposium's website.

Question: Are there participation fees ?

All EPTS activities are free of charge. Besides the symposium, there will be a banquet, to be held in Castel Toblino, a 12th century castle situated in the Toblino lake, in the evening of September 22. The banquet requires pre-registration and payment for the event and transportation. Details can be found in the registration form.
This is the Castle Toblino where the banquet is planned to be held.


Question: What is the notion of "active participation" ?

Unlike conferences where people come to hear presentations and maybe ask the presenters some questions, the notion of symposium is having the participants actively participate, this is done through the various panels that exist, which provide the possibility to present position statement, and discussions held with the audience. There are also some presentation opportunities. Below I am bringing an annotated plan of the symposium's session, annotating the opportunities for active participation, and then summarize the active participation opportunities.

Question: What sessions will be included in this year's program this year?

Here is an annotated list of the sessions. The meetings will last from 8:30AM to 6:45PM in the first and second day, and until 1PM in the third day.

The first day -- September 21 Monday.


Session I: EPTS - current state and further missions

We shall start (and end) the symposium by discussing the role of EPTS, what was achieved so far, and its missions for the future. This is an opportunity for active participation.

Session II: EPTS Use Case Working Group

This session will report the progress of the EPTS use case working group and will discuss the results and further activities.

Session III: Event Processing customers feedback

In this session we'll have a keynote address by a customer (TBD) and a panel made by customers providing feedback about the state of the practice and further directions from customers' point of view. This is an opportunity for active participation.


Session IV: Event Processing Research Contributions

In this session we'll hear several contributions from the research community about interesting research projects that are being held, there are already some requests for presentations, but there is a room for a couple of more presentations. This is an opportunity for active participation

The Second Day - September 22 - Tuesday

Session V: Reaching out – Event Processing and Business Process Management

During this symposium we shall have two sessions dedicating to "reaching out", dealing with the relationships between "event processing" as a discipline to other disciplines. The first of them is for relationships with the BPM area, following recent and planned discussions and meetings about even-driven BPM, the session will feature a keynote address by professor John Mylopoulos, a prominent researcher on the topic of Business Level Monitoring, followed by a panel that consists of people from both communities, and discussion with the audience. This is an opportunity for active participation

Session VI: EPTS Event Processing Languages Analysis workgroup

This session will report the progress of the EPTS languages analysis working group and will discuss the results and further activities.


Session VII: Reaching out – Event Processing and Network/Systems Management

This will be the second reaching out session for the area of network and system management which is a kind of event processing area that have been restricted to the boundaries of an application so far (see David Luckham's interview with Tom Bishop, the CTO of BMC).
The session will start with a keynote address of Dr. Kristian Stwart, the network availability architect in IBM Tivoli, formerly engineering director of Micromuse Netcool/Omnibus. This will be followed by a panel that consists of people from the two communities. This is an opportunity for active participation
Session VIII : Glossary and Interoperability session (tentative)

This session will deal with EPTS work groups about glossary and interoperability analysis, its format has not been determined yet. This is an opportunity for active participation


The Third Day - September 23 – Wednesday

Session IX: Architecture

This session will report the progress of the EPTS reference architecture working group and will discuss the results and further activities.


Session X: Grand Challenges

This is the closing session, it will feature a panel and open brainstorming about the grand challenges that are ahead of the event processing community, and discuss the preparation for the event processing Dagstuhl seminar planned for May 2010. This is an opportunity for active participation

An EPTS Business Meeting will follow on Thursday afternoon in the same place, but it is not an official part of the symposium.




Question: Can you summarize all the active participation opportunities ?


Here is the list:

1. Participation in the panel about EPTS missions.
2. Participation in the panel about customers feedback about the state of the practice.
3. Providing presentation in the event processing research projects presentation.
4. Participating in the panel about the relations between event processing and business process management
5. Participating in the panel about the relations between event processing and network/system management.
6. Participating in the panel about event processing grand challenges.

In addition, there will be opportunities for participating in open discussions.


Question: What do I need to do in order to participate in any of these activities?

If you are interested to participate in any of the panels or provide a research presentation, please let me know and I'll forward to the appropriate session chair; A session chair will be assigned soon and will be responsible for posing questions to the panel and participants, prior to the symposium, and moderate the session.

Question: What should I do now ?

If you want to participate, then use the symposium's website in order to register.

If you want to actively participate in any of the panel, then use Email to notify me about it.

If you know of anybody who might be interested to participate, and is not included in any of the notification forums and mailing list, then please forward him/her the link to this call for participation.

On the vision for smarter cities

The IBM Institute of Business Value has recently published its vision of smarter cities, which is part of the "Smarter Planet" vision on which I have written before. As noted before, the smarter planet vision is based on the fact that the world will be instrumented, interconnected and intelligent. Event processing is the backbone of smarter cities. As smarter cities is been labeled "system of systems", the integration and communication between the different systems is done using events, and the event processing filters, transforms and routes the event among systems. It also crates derived events from one or more systems that may be used by other systems. Event processing technologies have two roles: In the macro level, it is a glue between the systems and is part of the intelligent part of each individual system. I'll write more about some individual systems inside the system of systems. I am trying to get a smarter cities keynote talk in the 5th EPTS event processing symposium --- stay tuned, details about the symposium (Trento, September 21-23) are coming soon.

On hard coding event processing functionality

Today, again, I did not celebrate my birthday. I don't have a habit to celebrate a reminder to the fact that I am getting older. Interestingly, I got today much more (relative to recent years) birthday greetings in various communication ways -- E-Card through the Internet, phone calls, Email and even Real-time messaging. The reason that I got much more greetings relative to previous year can be attributed to the fact that I made many friends in the last year, but the more realistic reason is that certain social networks send people reminders about birthdays of other people in the network, which makes the knowledge about the birthday more accessible. Well -- none of my family remembered my birthday, here, at least, no change from previous years.

One of the many meetings I had last week was a teleconference with some IBM customers (I shall not expose their identity), my role in IBM is not in sales or sale support, but I am being called from time to time to participate in meetings with customers, by request of the people handling such meetings. The major issue that they wanted me to discuss is the benefit of using a middleware software for event processing (in the large sense) vs. hard coding it into the application code itself, like this customer is used to do.

Indeed, this is not a clear cut issue, there are two cases in which hard coding the event processing functionality make sense: either in the case that it is very simple, and thus it is not cost-effective to purchase, learn and deploy a specific product, or in the case that the functionality is too specific, and not covered by products, furthermore, it does not represent a ubiquitous requirement that makes it cost-effective for vendors to support it.

There are also some cases in which it is reasonable to write some functions in Assembly languages (even I did some of this in the past), but typically (most) people prefer to code in higher level language.

Likewise, in many applications, satisfy none of these conditions, and for these application is cost-effective to use generic software. The reason is that there are various common functions (e.g. filtering, routing, enrichment, transformation, pattern matching) that are repeating. Hard coding them means -- re-inventing wheels over and over again, instead of reusing existing implementations as "services", and enjoy other people's work that is being upgraded and optimized with time. This is similar to the reasoning of using other generic products -- messaging, workflows, databases, adapters, development environments and others.

The reaction of this particular customer was interesting: "what you are saying makes much sense, currently we are not used to think in terms of separating the event processing from the rest of the application logic, and we need to digest the idea". So, there will be a follow-up meeting to continue discuss it. I have seen this kind of reaction before, I think that a challenge of event processing vendors may be the competition with potential customers who do not understand the benefits above hard coding. But, this was also the case for other technologies that succeeded to cross this bridge. The growing number of event processing customers indicate that this thought is getting traction.

I think that this is also a topic for a community effort, that may be pursued by EPTS. More on this topic -- later.

More on generic vs. specific event processing

In the last few days I spent part of my time in the NGITS 2009 conference, that was hosted by the IBM Haifa Research Lab, today the keynote speaker was Alon Halevy, whose picture you can see above. Alon gave an excellent talk about "searching the structured Web". As a database person that joined Google 4 years ago he discovered that many of the things he knew from databases were not valid in Google. and from data management point of view, he went several steps forward, and his challenge was how to process structured data that is hidden in the "deep Web" behind web forms and interfaces. This reminded me one of my colleagues in the Israel Air force, who spent a year of his life more or less (almost around the clock) to write a proprietary database system based on low level I/O operations, for some logistics process (I think it was automatic warehouse), he wrote his version for conncurecny control, query capability, recovery, and many other stuff. He thought at that time that the DBMS products available in the market (this was 1978 I think) are not good enough, and using them will be a step backwards relative to what he had in mind. I have written in the past about single application vs. more general one in the context of why network and system management guys have not developed more general event processing products.

I was asked several times what is really the main issue behind all the work I am doing (along with many others) about "event processing" as a discipline, what is the new thing -- people have processed events forever. People also processed data long before DBMS has been introduced, but the way that my old colleague worked was not scalable. Not many people had the skills to do it, and it was not that cost-effective way.

Likewise, there were and still are many event processing applications of all types, colors and sizes, that are developed in an ad-hoc fashion, as there are still applications that process data that do not use DBMS products, because some aspect does not make it feasible or cost-effective.
However, it is clear that the DBMS area has made a tremendous contribution to the IT and business in the last few decades.

My own goal around event processing is to make event processing pervasive as part of enterprise computing, this will be achieved by generic software. Many of the database issues have been developed due the need for genericity -- take query optimization as an example, if one writes query by hand, it is not needed, since every developer can write optimized ad-hoc queries, the requirement to do query optimization came from the fact that a generic query language is being used for many purposes.

Personally, my interest is not in building "complex systems" (as the readers of my Blog know, I tend not to use the "CEP" acronym, due to the ambiguity of the term "complex") or one of kind systems. I think that the generic event processing systems will enrich their functionality over time, my interest is to make it pervasive. The first generation of products went some steps in this direction, and the next generations will do more. I have presented several times in various places my view about the next generation of event processing which refer to the challenges in the way to do it properly.

This topic will be discussed later this year in the 5th EPTS event processing symposium and the event processing Dagstuhl seminar planned for May 2010.