This is a blog describing some thoughts about issues related to event processing and thoughts related to my current role. It is written by Opher Etzion and reflects the author's own opinions
Showing posts with label DEBS 2011. Show all posts
Showing posts with label DEBS 2011. Show all posts
Thursday, September 22, 2011
What's in a title?
Some people noticed that I've changed my title in LinkedIn and sent me queries about it. I did not change my job today; as I noticed that my business cards are about to end and I need to issue a new one, I thought that this is also a good time to re-think on the content, and determined that the title I have there today does not really reflect what I am doing, so I switched to a better title, principle investigator of the Proton project, which is part of the IBM Research's "far reaching research" program (FRR), which we have extended, to explore the emerging paradigm of proactive event-driven computing. In the past I moved from working on concrete project to a more general technical strategy role, and then coming back to research, in working on requirements and challenges on the next generations of event processing, and towards the end of last year I moved back to concentrate on a single project which I believe has a huge potential. I have written about proactive computing before, and we exposed some of the ideas in the DEBS'11 paper.
I'll write more about this project in the future, and of course I am still writing about general event processing issues.
Monday, July 18, 2011
DEBS 2011 awards
Back in my office now from the DEBS trip, after spending Saturday in NYC and watched the matinee' show of Wicked, a wonderful musical.
Last remaining fact about DEBS 2011 is that it is the first instance of DEBS to grant awards, the award granting ceremony occurred at the conference banquet's on Wednesday evening. The awards are noted on the DEBS 2011 webpage. Here is the list of awards and award winners:
Best Paper Award: | Gabriela Jacques Da Silva, Buğra Gedik, Henrique Andrade, Kun-Lung Wu, Ravishankar K. Iyer. |
Fault Injection-based Assessment of Partial Fault Tolerance in Stream Processing Applications. | |
DEBS Challenge Award: | The ETH team:Lynn Aders, René Buffat, Zaheer Chothia, Matthias Wetter, Cagri Balkesen, Peter M. Fischer, Nesime Tatbul. |
Best Poster Award: | Nihal Dindar, Peter Fischer, Nesime Tatbul. |
DejaVu: A Complex Event Processing System for Pattern Matching over Live and Historical Data Streams. | |
Best Demo Award: | Sinan Sen, Ruofeng Lin, Bijan Fahimi Shemrani. |
Complex Event Pattern Evolution based on Real-Time Pattern Execution Statistics. | |
Best Idea in the DEBS Gong Show: | Mike Lefler. |
Saturday, July 16, 2011
On pattern rewriting - DEBS 2011 presentation
Among other things I have done in DEBS 2011, I have also delivered one talk to present the paper on patterns rewriting, the paper, like all DEBS 2011 papers can be obtained from the ACM Digital Library.
The paper is co-authored by Ella Rabinovich (the primary author), Avi Gal and myself.
The presentation starts by classifying optimization tools in use for optimizing the performance of event processing applications, the classification is into blackbox optimization where the actual implementation is taken as a blackbox, and whitebox optimization, The paper deals with one of the types of whitebox optimization - pattern rewriting, which mean rewrite a pattern into a collection of other patterns that yield equivalent results: same output to the same input.
The presentation can be viewed on slideshare. Enjoy!
Summary of DEBS 2011
In this picture you can see me explaining the rules of
the game for the "gong show", while Francois Bry, who was the gong master, is standing behind with the gong. You can also see that the podium has the Jeopardy! setting, and indeed we had in one of the sessions live game of two of the participants against Watson.
Paul Vincent, skilfully played the host, Scott Schneider and Michael Olson were the contestants, although the audience helped them Watson won this time also.
These pictures and many others were taken by Roland Stuhmer, and can be found on Flickr.
Besides the fun part we also had 33 papers, poster and demo session, DEBS challenges, tutorial day, PhD workshop and 4 keynote speakers
Overall, the community that includes researchers and some people from industry is becoming stable. the terminology becomes more coherent (everybody are talking about patterns), and the quality becomes higher.
There were participants from 23 countries from all the 5 continents including the first representative from Africa, a faculty member from Tanzania. I think that still the largest community is in Europe.
Organizing a conference is a lot of work, but is also a rewarding activity. My next conference in early August - AAAI in SFO, but there I am just a participant.
More on DEBS 2011 -- later
Tuesday, July 12, 2011
DEBS 2011 tutorial on non functional properties of event processing
This is the picture that opens the tutorial on non functional properties of event processing that has been delivered yesterday within the DEBS conference by Tali Yatzkar-Haham and myself. It shows variety of various cakes, each of them can be entitled: "cheese cake". Likewise, there is a substantial variance among event processing systems based on their non functional properties. We surveyed a collection of topics on performance, scalability, availabilty, usability and security/privacy. One of the conclusions is that the usability area lacks comprehansive research. At the end of the day we went (with some other friends) to eat dinner in the cheese cake factory in White Plains (I did not eat any cheese cake).
The tutorial is now available on slideshare - enjoy!
The tutorial is now available on slideshare - enjoy!
DEBS 2011 tutorial day
The first day of the DEBS conference is traditionally the tutorials day. There was a relatively large number of participants that attended this tutorials. I have listened to one tutorial given by Nenad Stojanovic and Pedro Bizzaro on the event processing grand challenge set in Dagstuhl, May 2010. I think that while all the ingredients are there, there still need to be some thinking about phrasing it in the grand challenge way, of an ambitious measurable goal. The second tutorial I attended (as a captive audience) was the tutorial that I presented together with Tali Yatzkar-Haham. I'll post this tutorial slides on slideshare, and write more about it within the next few days. Our tutorial covered the various issues of non-functional properties of event processing; there was some interesting discussion on several topics, like correctness, usability in general and transactional properties of event processing. Tomorrow the main part of the conference starts. More - later
Monday, July 11, 2011
Welcome to DEBS'11
After a year-long preparations, DEBS'11 is sailing out this morning. I have arrived yesterday to the USA, and now writing from the Westchester Marriott hotel in Tarrytown, which ACM selected as the hotel of residence for the conferences's participants. The conference itself will take place in the IBM Research headquarter in Yorktown Heights.
I am general chair of this conference, which meant a lot of work; the local organization chairs who took care of all logistics - Bugra Gedik and Gabriela Jacques Da Silva, have done excellent job, so hopefully everything will be smooth.
Looking forward to meet a lot of all friends, and some new ones, it seems that the event processing research community is growing, and that there will be quite a lot of industrial participants as well.
Besides the "back office" work, I'll have also some "front office" roles in the conference - starting today, I'll deliver a tutorial on non-functional properties of event processing, together with Tali Yatkar-Haham from my team, for her it will be the first participation in DEBS, another person of the team Yagil Engel is planned to present here as well. I'll also deliver one paper presentation in this conference (following Ella Rabinovich's thesis), and have some speaking roles in the opening session and in the banquet.
Will be extremely busy days for me, will also try to post some during the conference, I am sure it will also be covered by other bloggers (e.g. Paul Vincent).
More -later.
Monday, June 20, 2011
Plato vs. Aristotle
Carole-Ann Matignon recent Blog posting was entitled: "Plato or Aristotle", the two great Greek philosophers, that were once labeled as those in which the entire western culture is a footnote to their writings. In this context, I guess that Carole-Ann meant the major difference in their outlook of life. Plato saw the individual as part of a society, while his student Aristotle, saw the society as a collection of individuals. The difference is -- who is in the middle: the society or the individual. Carole-Ann's posting was in the issue of privacy, or data accumulated on people, which in some cases it is good for the society in general, a government agency, an ability to get credit decisions and more, but can harm the individual's interest. This is an issue that is also dominant in dilemmas about event processing for years, the relatively ease in obtaining information about events, in a world full of sensors and cameras, and the privacy considerations.
Security and privacy in event processing is one of the topics we'll discuss in the DEBS tutorial about non-functional properties of event processing in DEBS'11. Stay tuned for me.
Security and privacy in event processing is one of the topics we'll discuss in the DEBS tutorial about non-functional properties of event processing in DEBS'11. Stay tuned for me.
Friday, June 10, 2011
Call for participation in the DEBS'11 gong show
For everybody intending to participate in DEBS'11, I would like to attract your attention to a new track in DEBS - the gong show track, which is planned for Tuesday, July 12, after the reception, starting 5:15pm
Each speaker will have (strictly enforced by the gong) 5 minutes to talk about one of the following:
(1). Revolutionary idea to extend the state-of-the-art of event-based systems.
(2). Revolutionary use of event-based systems in the real-world.
There are limited amount of speaking slots and they will be determined on "first come first served" basis;
the audience will vote about the best idea, and the winner will be formally announced at the conference banquet on the next day.
If you are interested to present, please send Email to: debs2011@easychair.org with the title "gong show", your name, and optionally the title of your talk.
See you all in Yorktown in July.
Monday, May 30, 2011
Keynote talks in DEBS 2011
Titles and abstracts of the four keynote speakers in DEBS 2011 are now available, they will be updated on the DEBS website a little bit later, after the DEBS webmaster returns from vacation. All of them seem to be interesting talks.
Speaker: Chris Bird.
Chris is the solution architect for travel and leisure at Progress Software, until recently he has been the Chief Architect of Sabre Airline Solution.
Title: Avoiding he said/she said arguments in distributed event handling systems
Abstract: Processing critical event stream models, whether they be in chemical plants, in power stations, in process environments, in aircraft operations all have one thing in common. You want to know that the information got to where it was supposed to go, and you want to know that proper action was taken. Innocent sounding phrases, but with surprising complexity. Expand that thinking to conceptual business events and the complexity multiplies. In process control environments, for example, the payloads are typically small (10s to 100s of bytes), often in the form “Device id, Timestamp, Value1[,Valuen], so transmitting that data over a network, e.g. SCADA can be quite network efficient.However Business Events tend to be less frequent and will often have significant associated data destined for several “downstream” handlers. Taking an example from the airline industry, the event of rescheduling a flight has significant repercussions in Passenger Bookings, Airport Slot Management, Revenue Management, Equipment Scheduling, Catering, Fuel Fleet Scheduling, Crew Scheduling, together with knock on effects to other parts of the airline’s schedule as equipment is anticipated not to be available when planned. Add to that the extra complexity of having many other “businesses” involved. A late incoming flight may have effects on car rentals, hotel reservations, dinner reservations, etc. So the single event (a reschedule) has enormous ramifications. Along with these business ramifications, there are also legal ramifications. When a charge is to be disputed, (e.g. a hotel charging the first night fee because of a traveler no show) it is extremely important to know whether the hotel was notified prior to the cutoff time. The session introduces the ideas behind decoupling and situational awareness, so that efficient event handling takes place, together with appropriate situational awareness to enable policy decisions to be enacted.
Speaker: Don Ferguson
Don is Chief Technology Officer of CA.
Abstract: IT system and application management is critical to business use of IT systems. Distributed event processing is core to application and systems management, even for applications that are not "event driven." Emerging technology like virtualization and cloud computing significantly increase the central role of distributed event processing. IT systems and applications management introduces major challenges and requirements not typically seen in application centric event processing. This presentation provides an overview of IT system and application management use of distributed event processing, and the evolution for cloud computing. The presentation then provides an overview
of current solutions and technology to the requirements. Finally, there will be a discussion of open issues and research challenges.
Speaker: Johannes Gehrke
Johannes is a professor of computer science in Cornell University.
Title: Declarative Data-Driven Coordination
Abstract: There are many applications that require users to coordinate and communicate. Friends want to coordinate travel plans, students want to jointly enroll in the same set of courses,
and busy professionals want to coordinate their schedules. These tasks are difficult to program using existing abstractions provided by database systems because in addition to the traditional ACID properties provided by the system they all require some type of coordination between users. This
is fundamentally incompatible with isolation in the classical ACID properties of transactions.
In this talk, I will argue that it is time for the database and event processing communities to look beyond isolation towards principled and elegant abstractions that allow for communication and coordination between some notion of (suitably generalized) transactions. This new area of declarative data-driven coordination (D3C) is motivated by many novel applications and is full of challenging research problems. I will start by surveying existing abstractions in database systems and explain why they are insufficient for D3C. I will then describe entangled queries, a coordination language that extends SQL by constraints that allow for the coordinated choice of result tuples across queries originating from different users or applications, and I will discuss algorithms for evaluating entangled queries. I will conclude with a set of research challenges for event processing in this new area.
Speaker: Calton Pu
Calton is Professor and John P. Imlay, Jr. Chair in Software in the College of Computing,
Georgia Institute of Technology.
Title: A World of Opportunities: CPS, IOT, and Beyond
Abstract: The continuous evolution of computing and networking technologies (e.g., Moore’s Law) is creating a new world populated by many sensors on physical and social environments. This emerging new world goes much further than the original visions of ubiquitous computing and World Wide Web. Aspects of this new world have received various names such as Cyber Physical Systems (CPS) and Internet of Things (IOT). CPS links many physical sensor data to detailed simulation models running on large data centers. IOT brings together many appliances, making much more environmental data available and supporting control of these appliances. CPS/IOT applications are many, including personalized healthcare, intelligent transportation, smart grid, sustainable environment, and disaster recovery as representative examples. These CPS/IOT applications are motivated and strongly pushed by significant new social, economic, and human benefits. At the same time, these applications are also mission-critical with serious quality of service requirements such as real-time performance, continuous availability, high security and privacy. We will argue that the traditional process-oriented programming languages and software architectures should be augmented by distributed event-based facilities and abstractions for the construction of large scale distributed CPS/IOT applications. In addition to the focus on performance, we anticipate that other quality of service dimensions such as availability, reliability, security, and privacy will become important concerns in the research on distributed event-based systems. We will discuss research opportunities and challenges that bring the distributed event based systems technology to CPS/IOT applications.
Sunday, May 22, 2011
DEBS 2011 - call for participation
ACM DEBS 2011 CALL FOR PARTICIPATION
The ACM DEBS (Distributed Event Based System) is the flagship conference of the event-based systems community.
DEBS 2011 will be held in Yorktown Heights, NY, USA in July 11-15, 2011.
The DEBS 2011 program is now available in
http://debs2011.fzi.de/index.php/conference-programme The conference includes: * 23 Research papers * 10 Inudstrial papers * 5 Tutorials * Gong Show * Demo and poster workshop * 4 Keynote speakers:
Christopher Bird (chief architect, Sabre Airline Solutions);
Donald F. Ferguson (CTO, CA);
Johannes Gehrke (Cornell University);
Calton Pu (Georgia Institutue of Technology) * Special invited talk: Eddie Epstein (IBM Research) on Watson - the computer program that won against the Jeopardy! human champions
Registration site is open:
http://debs2011.fzi.de/index.php/conference-registration
Note that the early registation deadline is June 15.
See you all in Yorktown!
Wednesday, May 11, 2011
On big events and big data
The "Big Data" phenomenon gains a lot of traction, interest, and related work in recent years. The Internet and making everything in digital form has resulted in amounts of data beyond past imagination, and the rate of growth is amazing. Mark Palmer in his Blog posting made the analog of data as sand,
saying that "If every grain of sand in the bucket was 1 byte of data, then:
- The entire work of Shakespeare fills just one bucket of sand (about 5MB)
- A fast financial market data feed (OPRA) fills a beach of sand in 24 hours (about 5TB)
- Google processes all the sand in the world every week (about 100PB)
- We generate 60% more sand every year"
Events issue some of this data, but in many cases an event is the fact that a fact becomes true or false, and this fact is not really kept in the data.
The "Dagstuhl grand challenge", which is part of the event processing manifesto, is talking about an "event fabric", which will be the Internet equivalent of events instead of data, I guess that the quantities will be on the same cardinality, thus it will have the same scalability challenge. The main difference is the type of processing - event processing instead of queries/information retrieval. Getting to an "event fabric" has indeed many challenges. In DEBS 2011 there will be a tutorial about this grand challenge. I'll write more about this challenge in the future.
Tuesday, May 10, 2011
Watson meets DEBS 2011
DEBS 2011 will take place in the Yorktown Heights auditorium, the same site in which the famous recording to the Jeopardy! programs in which the computer has won against two human champions. The Watson hardware resides very close to this auditorium. With some effort we succeeded to bring Eddie Epstein, one of the senior persons behind Watson has agreed to be an invited speaker in DEBS 2011, Eddie will also provide a demo of the system.
The full program of DEBS 2011 will be advertised around this weekend, the list of accepted papers can be found on this Blog.
It should be note that if you plan to be in DEBS -- plan to stay for the entire conference, besides the 33 presentations, the attractions will be spread among the days - no day can be skipped!
Monday, July 11 will be the tutorial day, and in the late afternoon there will be the EPTS award granting event.
Tuesday, July 12 will have the two industrial keynote speakers: Chris Bird and Don Ferguson (see here for more about the keynote speakers), and in the later afternoon a reception, and after getting some alcohol, the "gong show", in which the participants can express outrageous ideas about future features and utilization of event processing technology.
Wednesday, July 13 will have the keynote talk of Johnnes Gehrke, the demo and poster session including the grand challenge, and the conference banquet with artistic performance, and the DEBS awards (best paper, best idea in the gong show, best demo and maybe more).
Thursday, July 14 will have the keynote talk of Calton Pu, and the invited talk of Eddie Epstein on Watson.
None of these day should be missed -- plan accordingly!
The fifth day - July 15 will be the PhD workshop adjacent to the conference.
See you all in Yorktown.
Saturday, April 30, 2011
DEBS 2011 - accepted papers and tutorials
DEBS 2O11 - here are the list of accepted papers and tutorials.
The registration site is now open:
Papers accepted to the Research Track
1 | Qiong Zou, Bugra Gedik and Kun Wang. | SpamWatcher: A Streaming Social Network Analytic on the IBM Wire-speed Processor |
2 | Nihal Dindar, Peter M. Fischer, Merve Soner and Nesime Tatbul. | Efficiently Correlating Complex Events over Live and Archived Data Streams |
3 | Richard Hull, Elio Damaggio, Riccardo De Masellis, Fabiana Fournier, Manmohan Gupta, Fenno Heath Iii, Stacy Hobson, Mark Linehan, Sridhar Maradugu, Anil Nigam, Piyawadee Sukaviriya and Roman Vaculin. | Business Entities with Guard-Stage-Milestone Lifecycles: Managing Entity Interactions with Conditions and Events |
4 | Jonas Buys, Vincenzo De Florio and Chris Blondia. | Towards context-aware adaptive fault tolerance in SOA applications |
5 | Jatinder Singh, David Eyers and Jean Bacon. | Disclosure control in multi-domain publish/subscribe systems |
6 | Ella Rabinovich, Opher Etzion and Avigdor Gal | Pattern Rewriting Framework for Event Processing Optimization |
7 | Yagil Engel and Opher Etzion. | Towards Proactive Event-Driven Computing |
8 | Amer Farroukh, Mohammad Sadoghi and Hans-Arno Jacobsen. | Towards Vulnerability-Based Intrusion Detection with Event Processing |
9 | Gianpaolo Cugola and Alessandro Margara. | High Performance Content-Based Matching Using GPUs |
10 | Rohit Wagle, Henrique Andrade, Kirsten Hildrum, Chitra Venkatramani and Michael Spicer. | Distributed Middleware Reliability and Fault Tolerance Support in System S |
11 | Michael Olson, Annie Liu, Matthew Faulkner and K. Mani Chandy. | Rapid Detection of Rare Geospatial Events: Earthquake Warning Applications |
12 | Vana Kalogeraki, Adam Dou, Dimitrios Gunopulos, Taneli Mielikinen and Ville Tuulos. | Scheduling for Real-Time |
13 | Sangjeong Lee, Youngki Lee, Byoungjip Kim, K. Selçuk Candan, Yunseok Rhee and Junehwa Song | . High-Performance Composite Event Monitoring System Supporting Large Numbers of Queries and Sources |
14 | Siddarth Ganesan, Young Yoon and Hans-Arno Jacobsen. | Take Five: The Management Infrastructure for Distributed Event-Driven Workflows |
15 | Bibudh Lahiri, Srikanta Tirthapura and Jaideep Chandrashekar | Space-efficient Tracking of Persistent Items in a Massive Data Stream |
16 | Amirhossein Malekpour, Antonio Carzaniga, Fernando Pedone and Giovanni Toffetti Carughi. | End-to-End Reliability for Best-Effort Content-Based Publish/Subscribe Networks |
17 | Eberhard Grummt. | Fine-grained Parallel XML Filtering for Content-based Publish/Subscribe Systems |
18 | Szabolcs Rozsnyai, Aleksander Slominski and Geetika Lakshmanan. | Discovering Event Correlation Rules for Semi-Structured Business Processes |
19 | Mark Linehan, Sylvain Dehors, Ella Rabinovich and Fabiana Fournier | Controlled English Language for Production and Event Processing Rules |
20 | Gabriela Jacques Da Silva, Bugra Gedik, Henrique Andrade, Kun-Lung Wu and Ravishankar K. Iyer. | Fault Injection-based Assessment of Partial Fault Tolerance in Stream Processing Applications |
21 | Xinxin Wang, K.Selcuk Candan and Junehwa Song | Complex Pattern Ranking (CPR): Evaluating Top-k Pattern Queries Over Event Streams |
22 | K R Jayaram and Patrick Eugster. | Program Analysis for Event-based Distributed Systems |
23 | Ming Li and Murali Mani. | Complex Event Pattern Detection over Streams with Interval-Based Temporal Semantics |
Papers accepted to the industry track
1 | Jimi Wen, Guyuan Lin, David Wu, Today Sung, Minsiong Liang and Gary Tsai. | A Complex Event Processing Architecture For Energy and Operation Management |
2 | Florian Kerschbaum. | Securely Disseminating RFID Events |
3 | Pekka Kaarela, Mika Varjola and Lucas Noldus. | PRONTO – Support for real-time decision making |
4 | Ashish Kulkarni. | |
5 | Alexandre Alves. | A General Extension System for Event Processing Languages |
6 | Ruediger Klein, Jingquan Xie and Andrij Usov | Complex Events and Actions to Control Cyber-Physical Systems |
7 | Naomi Seyfer, Richard Tibbetts and Nathaniel Mishkin. | Capture Fields: Modularity in a Stream-Relational Event Processing Language |
8 | Hannes Obweger, Josef Schiefer, Martin Suntinger, Peter Kepplinger and Szabolcs Rozsnyai. | User-Oriented Rule Management for Event-Based Applications |
9 | Pål Evensen and Hein Meling. | A Paradigm Comparison for Collecting TV Channel Statistics from High-volume Channel Zap Events |
10 | Nenad Stojanovic, Dejan Milenovic, Yongchun Xu, Ljiljana Stojanovic and Darko Anicic. | An intelligent event-driven approach for efficient energy consumption in commercial buildings: smart office use case |
Accepted tutorials
1 | Gianpaolo Cugola and Alessandro Margara | Processing Flows of Information: From Data Stream to Complex Event Processing |
2 | Opher Etzion and Ella Rabinovich | Non Functional properties of event processing |
3 | Chathura Herath | Programming abstraction for Event processing in e?sciences – Dilemma of managing high data event rates with high resource consuming computations |
4 | Nenad Stojanovic, Pedro Bizzaro and Mani Chandy | Event Processing Grand Challenges |
5 | Scott De Deugd and Dave Locke | Open Messaging in the Real World |
6 | Paul Vincent, Adrian Paschke, Catherine Moxey and Alexandre Alves | Architectural and Functional Design Patterns for Event Processing |
Subscribe to:
Posts (Atom)