Startup review: Hstreaming

Late last week I got a briefing on a new startup  - HSTREAMING.   The briefing was delivered by an ex-colleague in IBM Research, Volkmar Uhlig.   The idea behind HSTREAMING is providing Hadoop-based platform that enables running aggregations, filtering and some forms of event pattern matching in real-time.  The idea is that since there is a growing use in Hadoop,  Hadoop-based applications, which is batch-oriented, will be developing more and more extensions that require online processing along with the batch processing.  This is the Hadoop variation of using database and stream processing together.    Certainly and interesting direction; I think that we are seeing variations of MapReduce coupled with event processing in various places.  I'll continue to follow this direction.   

La vita è bella

I have a tradition to write about the holocaust day, two years ago I have written about my father and the king Alfred,  a posting that brought me many reactions from around the world.   Today is the eve of the holocaust day in Israel, and the TV broadcasts all evening holocaust related programs an movies.   My daughters called me to watch a movie called: La vita è bella, an Italian speaking movie  (in Israel the movies are not dubbed, they are sub-titled talking about father and son, both Italian Jewish, sent to a concentration camp, and the father plays for the son as if everything is one big game, and saves him various times.   It is a great movie (three academy awards),  where besides the plot there is a lot of symbolics, the idea to view all the events in the concentration camp as a game, where in some points the child discovers the reality, and tells his father that he heard that they are doing buttons and soap from people, and burning them, and his father treats it as if it is all just a joke.   
As I have written before, my late father, who was a sole survivor of a big family, used to say that this period occurred on another planet, and people who were not there, cannot understand it.   I think that it is good that one day a year is dedicated here to educate the younger generation which is two generations removed and give them some glance on that dark period. 

Event processing and Erlang

These two pictures are taken from the recent family trip to Barcelona.  In the second one you can see also my wife and my daughters Adi and Daphna (my daughter Hadas is the photographer).  Daphna also uploaded many pictures of the trip to Facebook  (with tagging in Hebrew). 

Recently I came across a posting in Streambase's  Blog,  talking about embedding Erlang in the Streambase development environment, under the title: Streambase loves Erlang.   It is worth mentioning that there have been some views about the use of Erlang in event processing before,  first by Marco Sierio in the Rulecore Blog, a while ago, and a related presentation talking about the use of Erlang in inference rules, was presented by SAP Research.  

Erlang is actually quite an old  language, a functional language that is aimed at massively concurrent applications.   Personally as an old-timer, I like functional programming more than object-oriented imperative languages,  and happy to see a revival of such languages.   It might be interesting idea to write event processing functionality using Erlang.  I am toying with the idea to play with it by means of students projects' next time I am teaching event processing course (in the fall).   I may ask for people who have done it for best practices.     

DEBS 2011 - accepted papers and tutorials

DEBS 2O11 -  here are the list of accepted papers and tutorials.
The registration site is now open:  

Papers accepted to the Research Track

1	Qiong Zou, Bugra Gedik and Kun Wang.	SpamWatcher: A Streaming Social Network Analytic on the IBM Wire-speed Processor
2	Nihal Dindar, Peter M. Fischer, Merve Soner and Nesime Tatbul.	Efficiently Correlating Complex Events over Live and Archived Data Streams
3	Richard Hull, Elio Damaggio, Riccardo De Masellis, Fabiana Fournier, Manmohan Gupta, Fenno Heath Iii, Stacy Hobson, Mark Linehan, Sridhar Maradugu, Anil Nigam, Piyawadee Sukaviriya and Roman Vaculin.	Business Entities with Guard-Stage-Milestone Lifecycles: Managing Entity Interactions with Conditions and Events
4	Jonas Buys, Vincenzo De Florio and Chris Blondia.	Towards context-aware adaptive fault tolerance in SOA applications
5	Jatinder Singh, David Eyers and Jean Bacon.	Disclosure control in multi-domain publish/subscribe systems
6	Ella Rabinovich, Opher Etzion and Avigdor Gal	Pattern Rewriting Framework for Event Processing Optimization
7	Yagil Engel and Opher Etzion.	Towards Proactive Event-Driven Computing
8	Amer Farroukh, Mohammad Sadoghi and Hans-Arno Jacobsen.	Towards Vulnerability-Based Intrusion Detection with Event Processing
9	Gianpaolo Cugola and Alessandro Margara.	High Performance Content-Based Matching Using GPUs
10	Rohit Wagle, Henrique Andrade, Kirsten Hildrum, Chitra Venkatramani and Michael Spicer.	Distributed Middleware Reliability and Fault Tolerance Support in System S
11	Michael Olson, Annie Liu, Matthew Faulkner and K. Mani Chandy.	Rapid Detection of Rare Geospatial Events: Earthquake Warning Applications
12	Vana Kalogeraki, Adam Dou, Dimitrios Gunopulos, Taneli Mielikinen and Ville Tuulos.	Scheduling for Real-Time Mobile MapReduce Systems
13	Sangjeong Lee, Youngki Lee, Byoungjip Kim, K. Selçuk Candan, Yunseok Rhee and Junehwa Song	. High-Performance Composite Event Monitoring System Supporting Large Numbers of Queries and Sources
14	Siddarth Ganesan, Young Yoon and Hans-Arno Jacobsen.	Take Five: The Management Infrastructure for Distributed Event-Driven Workflows
15	Bibudh Lahiri, Srikanta Tirthapura and Jaideep Chandrashekar	Space-efficient Tracking of Persistent Items in a Massive Data Stream
16	Amirhossein Malekpour, Antonio Carzaniga, Fernando Pedone and Giovanni Toffetti Carughi.	End-to-End Reliability for Best-Effort Content-Based  Publish/Subscribe Networks
17	Eberhard Grummt.	Fine-grained Parallel XML Filtering for Content-based Publish/Subscribe Systems
18	Szabolcs Rozsnyai, Aleksander Slominski and Geetika Lakshmanan.	Discovering Event Correlation Rules for Semi-Structured Business Processes
19	Mark Linehan, Sylvain Dehors, Ella Rabinovich and Fabiana Fournier	Controlled English Language for Production and Event Processing Rules
20	Gabriela Jacques Da Silva, Bugra Gedik, Henrique Andrade, Kun-Lung Wu and Ravishankar K. Iyer.	Fault Injection-based Assessment of Partial Fault Tolerance in Stream Processing Applications
21	Xinxin Wang, K.Selcuk Candan and Junehwa Song	Complex Pattern Ranking (CPR): Evaluating Top-k Pattern Queries Over Event Streams
22	K R Jayaram and Patrick Eugster.	Program Analysis for Event-based Distributed Systems
23	Ming Li and Murali Mani.	Complex Event Pattern Detection over Streams with Interval-Based Temporal Semantics

Papers accepted to the industry track

1	Jimi Wen, Guyuan Lin, David Wu, Today Sung, Minsiong Liang and Gary Tsai.	A Complex Event Processing Architecture For Energy and Operation Management
2	Florian Kerschbaum.	Securely Disseminating RFID Events
3	Pekka Kaarela, Mika Varjola and Lucas Noldus.	PRONTO – Support for real-time decision making
4	Ashish Kulkarni.	ARCADE - ABSTRACTION AND REALIZATION OF COMPLEX EVENT SCENARIOS USING DYNAMIC RULE CREATION
5	Alexandre Alves.	A General Extension System for Event Processing Languages
6	Ruediger Klein, Jingquan Xie and Andrij Usov	Complex Events and Actions to Control Cyber-Physical Systems
7	Naomi Seyfer, Richard Tibbetts and Nathaniel Mishkin.	Capture Fields: Modularity in a Stream-Relational Event Processing Language
8	Hannes Obweger, Josef Schiefer, Martin Suntinger, Peter Kepplinger and Szabolcs Rozsnyai.	User-Oriented Rule Management for Event-Based Applications
9	Pål Evensen and Hein Meling.	A Paradigm Comparison for Collecting TV Channel Statistics from High-volume Channel Zap Events
10	Nenad Stojanovic, Dejan Milenovic, Yongchun Xu, Ljiljana Stojanovic and Darko Anicic.	An intelligent event-driven approach for efficient energy consumption in commercial buildings: smart office use case

Accepted tutorials

1	Gianpaolo Cugola and Alessandro Margara	Processing Flows of Information: From Data Stream to Complex Event Processing
2	Opher Etzion and Ella Rabinovich	Non Functional properties of event processing
3	Chathura Herath	Programming abstraction for Event processing in e?sciences – Dilemma of managing high data event rates with high resource consuming computations
4	Nenad Stojanovic, Pedro Bizzaro and Mani Chandy	Event Processing Grand Challenges
5	Scott De Deugd and Dave Locke	Open Messaging in the Real World
6	Paul Vincent, Adrian Paschke,  Catherine Moxey and Alexandre Alves	Architectural and Functional Design Patterns for Event Processing

Briefing a USA federal committee

In this picture you can see me and a tiger during my family vacation in Thailand, in 2007.
Today I met another TIGER - the standing committee for Technology Insight-Gauge, Evaluate & Review which is a committee of the USA National Academics that works on behalf of the USA intelligence community. 

I am still unsure how they picked me up, but they invited me to participate in a meeting with the committee members and some other participants through video conference and brief them on the state of the practice, my vision for event processing in 2020, and the challenges on the way to get there.  This is an indication that event processing has caught the attention of  defense sector as well.    The vision part talked about the Event Fabric devised as the grand challenge in the Dagstuhl event processing seminar,  published as chapter 5 in the event processing manifesto (BTW - in DEBS 2011 there will be a tutorial about this grand challenge). 
Another topic was of course, the proactive world, my favorite topic.  


I was asked what is the killer application that will drive the 2020 vision, and said there is no single killer application, there are multiple of them, and I see many of them outside the corporate IT domain, such as autonomic robots and computing embedded in biological systems.   Of course, defense applications will also be part of the driving forces.    More on this topics -- later

DEBS 2011 news

The DEBS 2011 track chairs have sent acceptance (and rejection) notes to the authors.  This time there has been substantial growth in the number of submissions, and also seems that there were quite a lot of quality papers, not all of the high quality papers could be accepted.


Some statistics: 


 The research track accepted 23 papers 
 The industry track accepted 10 papers  
 The tutorial track accepted  6 tutorials 


PhD workshop notifications will be made shortly.   Accepted papers co-authored by PhD students qualify the students also the right to participate in the PhD workshop and get feedback in a mentoring atmosphere by some of the leading figures of the area.


Next call is the posters and demos call - open until May 2nd.   This gives a chance to present work in progress, and also provides opportunity to authors of rejected papers to present a poster and interact with the participants on their research.

On predictive and proactive processing

Recently, there was a notification about IBM Research project, from the Almaden Research Center, that deal with prediction of traffic jams.  Prediction of traffic jams is always useful,  as you don't want to get caught by surprise,   this afternoon we are going to celebrate the holiday in the traditional family "Passover Seder".  The radio already predicts that all routes will be loaded,  which is good to prepare yourself for a long stay on the road, but don't help much in resolving it.    Prediction is an important step, but to complete the loop we need also the proactive behavior, in which one can also do something about it.    The proactive solution may be to go via another route, which may not be helpful today - since all roads will be jammed.   Anyway - the ability to predict future events is a key aspect in proactive computing, and they are used as a basis for proactive decisions. 


Going also tomorrow morning for a few days vacation in Barcelona.   Back - next week.