Packing and about to leave the spartan room in the Dagstuhl Schloss towards the last session. Today at noon the Dagstuhl seminar will be over, and the easier part of the mission will be complete, now we'll have to finish the document and devise the follow-up action items and mechanism to track them. I'll summarize the seminar later-- yesterday there were deep dives on standards and on the issue of relations between event processing to other areas and disciplines. We also had evening session with Alex Buchmann as moderator, about the question -- whether "event processing" has become a research community, is the flagship conference of the community DEBS succeeding, and how we would like to evolve it? should we do ACM SIG and when? should we have a publication like Sigmod Record and when? should we have an academic journal and when? it seems that we are on track, still need to recruit more people whose participation can gain DEBS a status of top conference that don't send paper to DEBS since they are sending just to top conferences, so it is a chicken and egg issue, and we have a challenge to reach out to adjacent communities. We'll take it into account when designing DEBS 2011. This is a blog describing some thoughts about issues related to event processing and thoughts related to my current role. It is written by Opher Etzion and reflects the author's own opinions
Friday, May 21, 2010
Dagstuhl seminar on event processing - the fourth day
Packing and about to leave the spartan room in the Dagstuhl Schloss towards the last session. Today at noon the Dagstuhl seminar will be over, and the easier part of the mission will be complete, now we'll have to finish the document and devise the follow-up action items and mechanism to track them. I'll summarize the seminar later-- yesterday there were deep dives on standards and on the issue of relations between event processing to other areas and disciplines. We also had evening session with Alex Buchmann as moderator, about the question -- whether "event processing" has become a research community, is the flagship conference of the community DEBS succeeding, and how we would like to evolve it? should we do ACM SIG and when? should we have a publication like Sigmod Record and when? should we have an academic journal and when? it seems that we are on track, still need to recruit more people whose participation can gain DEBS a status of top conference that don't send paper to DEBS since they are sending just to top conferences, so it is a chicken and egg issue, and we have a challenge to reach out to adjacent communities. We'll take it into account when designing DEBS 2011. Saturday, May 2, 2009

Packing for a one week (net) travel to the USA. We had recently been informed that the paper entitled: A stratified approach for supporting high throughput event processing application
Here is the abstract of this paper:
financial applications in large banks are processing 400 million events per day, and Massively Multiplayer Online (MMO) games are monitoring in peaks 1 million events per second. It is evident that scalability in event throughput is a major requirement for these types of pplications. While the first generation of event processing systems has been centralized, we see various solutions that attempt to use both scale-up and scale-out techniques. Alas, partitioning of the processing manually is difficult due to the semantic dependencies among various event rocessing agents. It is also difficult to tune up the partition dynamically in a manual way. Manual partitioning is typically vertical, i.e. there is a single partition set with centralized routing. This paper proposes a horizontal partition that is automatically created by analyzing the semantic dependencies among agents using a stratification principle. Each stratum contains a collection of independent agents, and events are always routed to subsequent strata. We also implement a profiling-based technique for assigning agents to nodes in each stratum with the goal of aximizing throughput. A complementary step is to distribute the load among the different execution nodes dynamically based on performance characteristics of nodes and agents and the event traffic model. Experimental results show significant improvement in the ability to process high hroughput of events relative to both centralized solutions as well as vertical partitions. We find this to be a promising approach to achieve high scalability without requiring difficult manual tuning, especially when the traffic model and the topology of the event processing network is often changed.
More about event processing distribution and parallelization will be discussed in subsequent postings.
DEBS has also issued recently a call for fast abstracts, posters and demos, an opportunity to share with the community work that is in less mature phase. show interesting demos, and discuss ideas.
Sunday, February 22, 2009
On Event Processing and Software Life-Cycle Processes
I'll start with a reminder that the deadline for submitting papers for DEBS 2009. The DEBS conference is the major conference for event processing related research. I would like to highlight especially the industrial track and solicit submission both from vendors and customers. Vendors may not submit a marketing oriented paper, however can send papers about architectures, technical questions, language issues, semantic issues, and engineering related topics. We especially solicit experience reports of event processing systems of any kind that are working in production, with experience insights that may be of general interest. Note that a short abstract is due next Monday, February 23rd, the deadline for submission is currently March 2nd, however extension of a few days will probably be possible.
Many years ago I have taken a course that has qualified me to be "IT project manager", and when talking about software life-cycles the instructor showed us the picture above, since then I have a Pavlovian Conditioning that the term life-cycle is associated with this picture.As a follow up to my recent posting about static and dynamic event flows, one of the related questions is whether event processing systems have the same life-cycle as other software artifacts, i.e. has to go through development and testing processes before going into production, or since it is very easy to modify such applications, this is not really needed, and changes can be done fast.
As usual in event processing there is no single answer. In some cases the event processing software is part of mission critical applications and thus any change should go through some validation and approval processes, which, alas, takes time and hurts the agility; in the other extreme there are systems in which an individual consumer defines personal alerts from events related to social networks, this application is not part of any enterprise application, and the consumer can add/modify/delete alerts every 10 minutes without upsetting anything except for himself, so there is no meaning to life-cycles and red-tapes.
Some applications may be hybrid. Part of it is a controlled logic, and part of it can be free for use to users.
Tuesday, November 4, 2008
On some EP related conferences

As I said all I had to say about the EDA vs. CEP issue, and does not wish again to get inside discussion of "what is CEP", I'll write today about some conferences in the event processing area. In the top of this page I've put the logo of DEBS 2009 which became the flagship research conference of the event processing community. The conference that has become ACM conference will include, like last year, industrial track that will contain - experience reports, architectures, interesting applications, and anything else of interest from the industrial perspective. We'll also try to solicit intermediate reports of the EPTS work-groups. Follow the call for papers in the conference website.
Another conference that announced extension in submission : the AAAI symposium on intelligent event processing - submission extended to November 13th.
Happy conferencing.
Saturday, July 12, 2008
On messages and events
Wednesday, July 9, 2008
On the multiple types of patterns in event processing

In this slide there are four possible meanings for patterns in event processing:
Saturday, July 5, 2008
On events and Data

Friday, July 4, 2008
On DEBS 2008
- Boundaries and territories among disciplines -- in the wake of discussion with Dieter Gawlick from Oracle.
- The term "event processing agents" - some discussions, and Blog comments from Paul Vincent
- The notion of pattern in event processing.
Overall - very good conference, an opportunity to meet a lot of friends, and (I am ashamed to admit) my first visit in Rome ever.
Monday, June 30, 2008
On the EPTS working group use cases
Hello from hot and humid Rome, Italy. I have just finished eating a Pizza dinner just across the street from the Coliseum, seen in this picture, lot of ancient stuff around. I am now writing this Blog in the hotel lobby - just 5 minutes walk from there, why am I writing from the hotel lobby? One of my criteria to chose hotels is that they advertise that they have Internet access in their rooms, and this hotel also claim to have Internet access - nominally they have , it is just that the signal in the rooms is too weak, and in order to work one has to come to the lobby - which is an information that the receptionist gladly shares with you after you check in... Anyway, they have here a comfortable desk to work and air-condition, which is more than we had today in the university, where we had been meeting for the EPTS working group. So - to start from the beginning, I have arrived to Rome last night for the DEBS conference. If you look at the DEBS site you can discover that the air-condition in the university has been broken, and they move the conference to anther site -- well, but just from Wednesday, and tomorrow we'll have to suffer another day in the non air-conditioned building. This has been the first time in history that I have closed my laptop, since it got so hot that I feared it will not survive. Tomorrow I'll give a tutorial in front of a sweating audience just after lunch, it will be fun. Today, we had a F2F meeting of the use cases working group - interesting mix of four people from different vendors (Dieter Gawlick from Oracle, Richard Tibbetts from Streambase, Brian Connell from WestGlobal and myself), one academic person (Pedro Bizzaro) and one customer (Alex Kozlenkov from BetFair), there are some other members that could not participate, the working group is larger that the six of us. Despite the heat it has been long and productive day -- what are we trying to achieve? We are trying to learn something from the variety of use cases out there, we have heard until now in the three EPTS meetings and Dagsthul seminar, something like 80 use case. However, we cannot learn much of them, since they have been presented in an unstructured way, we could not analyze them on the same scale - the purpose is to try and classify event processing applications to several classes - and observe what are the important requirements for each class - in terms of functional, non-functional requirements, ROI measurements etc..., this is done to enable existing and potential customers to better understand the types of event processing (complex event processing or other event processing types) - today we have worked on the template - what are the criteria to analyze each individual use case, and got to a draft there that will be finalized in the next few weeks. We shall present the template together with some use cases on this template in the next EPTS F2F meeting in September - so stay tuned. Tomorrow -- DEBS. While DEBS is an academic conference, I am glad to see here many of my industrial colleagues (both vendors and some customers), the establishment of a research discipline around event processing, and help growing the academic research in this area, has been one of the EPTS basic missions, and we are supporting DEBS as the research academic flagship. Will write my impressions from the conference later this week (if my laptop will survive the heat)...
Wednesday, December 26, 2007
EPTS plans for 2008

This picture has been taken in the EPTS meeting in Orlando in September 2007, the meeting itself has been described by Paul Vincent in his blog : first day, second and third day.