My talk in DEBS 2014 on the Internet of Everything

I am writing this post from the hotel "Meluha the Fern" in Mumbai.  Arrived here on Friday and had also an opportunity to do some sightseeing. Will write my impressions from Mumbai at a later phase.

Today DEBS 2014 started, the conference is being held in IIT Bombay.   The first day has been the tutorial day. I have delivered (by myself, my co-authors did not arrive) a tutorial on the "Internet of Everything".

This is the next in the tradition of tutorials that I am giving in DEBS since 2008.   As usual I have posted the tutorial on slideshare.  The problem with the slideshare conversion is that it messes up the animations, but I guess that it is readable anyway.  I'll write about the rest of the conference soon.  Enjoy!

Start of a new semseter and upcoming talks

I have not written recently -- some due to a short vacation (in Paris), and some due to seasonal laziness.. 

In any event - this week the semester started in Israel, and I am giving an Event Processing course again at the Technion, this time I am doing some change in the course.   The course is still based on the EPIA book,  but I am updating now to teach event processing through a model-based approach, which corresponds to the project we are working on recently.  We are going to do first public exposure of this work in ER 2013 in Hong-Kong.  I'll write about it later.  

Next week, I am giving keynote talk in an event oriented workshop co-located with ACM Multimedia 2013.  in which I'll talk about the semantics and modeling of situations and contexts.   Both talks will become public after they are given.   

Our DEBS 2013 tutorial: why is event-driven thinking different than traditional thinking about computing?

The tutorial that Jeff Adkins and myself delivered in DEBS 2013 yesterday is now posted on slideshare.  

Many of the participants reacted with further discussions.  The first take away we wanted to convey is described on this slide.   While the community continues to work on advancing the technology (which is fine!), the main challenge today is not the technology,  but making the paradigm shift in people's mind about 

event-driven thinking.  Indeed, experience shows that a typical developer when facing with an application which we classify as event-driven would tend to implement it in a traditional way:  treat the events as data, store them in a database, and process them in request/response mode.   The tutorial discusses the differences and explains the ontology of event-based system as we see it.  It also has a short description of our recent work around the "Event Model", on which I'll write more in the future.  

More about DEBS 2013  --- soon. 

Prelude to the DEBS 2013 tutorial

The notable event of this week in our family was the wedding of our eldest daughter Anat. In the picture you can see Anat, her new husband Adi with my wife and myself, in fancy dresses.

It was also very loaded week at work, one of the items has been submitting extended abstract of the  tutorial that Jeff Adkins and myself are  planned to deliver in DEBS 2013.     So here is the tentative outline we submitted in the original proposal:

Outline  

Topic I:  Introduction – Brief history of event processing  

Topic II:  The major differentiation factors of event-based thinking

Topic III:  The ontology of events and event influence

Topic IV:  Anatomy of reactive systems 

Topic V: Pragmatics – a computational independent model for event-based systems 

The tutorial starts with follows some of my previous postings about "event oriented thinking",   and Jeff's 4D classification.  It analyzes the way that people think about systems in the conventional way vs. the way people think in event-driven fashion, and gets into the role of events in language, and in system modeling.  

Unlike the implementation oriented thinking we usually employ -- this is a computational independent thinking, it looks at the event-based systems from the customer's perspective.     See (at least some of) you in Arlington.

DEBS 2013 - keynotes and tutorials were published

DEBS 2013 will take place in the campus of the University of Texas Arlington, in June 29- July 3.

Today the keynotes and tutorials were published on the conference's website:

The keynote speakers this time will be: Roger Barga from Microsoft, one of the first persons in Microsoft Research who worked on event processing, and crossed the line to the product organization and deals now with product management.  Roger will talk about "the rise of the velocity pipeline in enterprise computing" that will focus on the velocity part of big data that makes batch solutions like Hadoop as inadequate.

The second keynote speaker is David Wollman who manage the smart grids standard activities in  NIST  will talk about smart grids.

There will be four tutorials - all of them of known people active in the area.

The first one will be given by my IBM colleagues from the System S team who will talk about "stream processing optimizations".  

The second one will be given by Christoph Emmersberger  an Florian Springer (both of them I know from their past association with Rainer von Ammon), who will talk about the event processing capabilities of Apache Camel.

The third one will also be given in German accent, by Boris Koldehofe and  Frank Dürr from University of Stuttgart. They will talk about "Software defined networks".

Last -- keeping the tradition, I'll be giving a tutorial this year also, this time together with Jeff Adkins, on a topic that we are both dealing - "why is event driven thinking different from regular thinking about computing". I'll write about this tutorial at a later phase (well, we have to prepare it first), meanwhile you can read the short abstract on the site.  Hope to meet old friends and colleagues in Arlington.   

More - later.

DEBS 2012 tutorial on event processing design patterns

In the morning I had to attend the tutorial that I have co-presented,  but in the afternoon I was free to chose among three tutorials.  Out of sheer laziness I stayed in the same room and listened to the tutorial given on behalf of the EPTS reference architecture workgroup.   The gang of four that presented this tutorial were:

• Paul Vincent (who gave his performance wearing a papillon  tie - I hope that somebody took a picture of him, I have put a picture of some papillon tie as a place holder)

      Paul was the main presenter, and also presented the TIBCO interpretation and the Oracle one (covering  

      for Alex Alves who couldn't arrive).

• Adrian Paschke - who presented the Prova interpretation (Prova is an open source that he co-developed)
• Catherine Moxey - who presented the IBM Websphere Operational Decision Management (WODM) intepretation 
• Martin Hirzel - who represented IBM Infosphere Streams

The tutorial was uploaded to the DEBS 2012 site along with the other tutorials  

I need to review this tutorial carefully in order to comments,  but several comments off-hand;

First - I think that the team has done a very thorough work of trying to classify functions, map them to various implementations, they also attempted to map them to the EPIA book that Peter Niblett and myself have composed.   

I need to think more carefully about some terminology issues --- first, I am not sure that what they presented are really design patterns as defined in software engineering by the original gang of four.  I think that what was done is more classification of functions not really design practices.  Their classification is to four major classes: event reaction, complex event detection, event analysis and event preparation. 

The complex event detection contain three functions:  consolidation, composition and aggregation.  I am not really sure that the definitions are intuitive, but I am taking as an action item to look at the definition and right a detailed review of them.   There are some other stuff that worth reading and reviewing, including validating the relationships that they made with the EPIA patterns (again in EPIA these are functions and not patterns).

The mapping to the different languages is also very interesting.  Work on real best practice of patterns and anti-patterns will be a good next step.

More -later

DEBS 2012 tutorial on uncertainty in event processing

Some people react badly to uncertainty, as seen in the illustration above,  some people are very good in dealing with uncertainty,  I am still uncertain to which of these two categories I belong.

Today I have delivered a tutorial on uncertainty in event processing,  in the tutorial day that precedes the main conference of DEBS.   

The tutorial was 3 hours, I delivered around half of it, and the other half was given by Alex Artikis.  Fabiana Fourier and Zohar Feldman from our team in IBM Haifa Research Lab participated in the preparation.         I have written earlier this year about the issue of uncertainty in event processing.   In the tutorial we provided some motivation, talked about the representation of uncertainty, handling uncertainty, and Alex contributed his part by talking about different AI techniques to uncertainty event handling.  I'll write a series of posts about the different chapters of this tutorial in a later phase, but meanwhile you can view this presentation on slideshare.    I'll write more on DEBS 2012 later.

New tutorial on event processing

Roland Stuhmher, whom you can see in the right-hand side of the picture, on the Jeopardy! set, in DEBS'11, has recently recorded a video piece giving a tutorial on event processing.  Some of the slides looks familiar to me (well, he mentioned my name is his slide about "attribution of the slides").   In the slides he mentions a term iCEP, which either means that Apple has a new CEP gadget in its i series.  BTW - according to Steve Jobs, the apple "i" (started with iMAC) stands for: Internet, Individual, Instruct, Inform, Inspire -- all start with I...  So does any of them apply to iCEP?  Another possibility is, of course, intelligent (I have used the term IEP in the past). Anyway -  good tutorial, with some glance of the ETALIS project developed in FZI.  Enjoy!

AAAI 2011 tutorial on event processing and its challenges

San Francisco.   Just had dinner in an English pub here, it also serves the same food of pubs in England, and the receptionist at the hotel is saying "lift" instead of "elevator",  but I am really not in England, but in San Francisco.  

Today I have given a tutorial together with Yagil Engel in AAAI 2011 about event processing and research challenges in establishing the next generations of event processing that might be of interest to the AI community. 
The tutorial is now available on slideshare.  
 It consists of two parts - the first one is introduction to event processing which is similar to previous talks I have given, and is in essence a short version of the VLDB 2010 tutorial,  the second part talked about the challenges,  the outline slide of the second part can be seen below:

More details in the tutorial itself -- enjoy!

DEBS 2011 tutorial on non functional properties of event processing

This is the picture that opens the tutorial on non functional properties of event processing that has been delivered yesterday within the DEBS conference by Tali Yatzkar-Haham and myself.    It shows variety of various cakes, each of them can be entitled: "cheese cake".     Likewise, there is a substantial variance among event processing systems based on their non functional properties.  We surveyed a collection of topics on performance, scalability, availabilty, usability and security/privacy.   One of the conclusions is that the usability area lacks comprehansive research.     At the end of the day we went (with some other friends) to eat dinner in the cheese cake factory  in White Plains (I did not eat any cheese cake). 


The tutorial is now available on slideshare -  enjoy!

DEBS 2011 tutorial day

The first day of the DEBS conference is traditionally the tutorials day.  There was a relatively large number of participants that attended this tutorials.  I have listened to one tutorial given by Nenad Stojanovic and Pedro Bizzaro on the event processing grand challenge set in Dagstuhl, May 2010.  I think that while all the ingredients are there, there still need to be some thinking about phrasing it in the grand challenge way, of an ambitious measurable goal.    The second tutorial I attended (as a captive audience) was the tutorial that I presented together with Tali Yatzkar-Haham.    I'll post this tutorial slides on slideshare, and write more about it within the next few days. Our tutorial covered the various issues of non-functional properties of event processing; there was some interesting discussion on several topics, like correctness, usability in general and transactional properties of event processing.    Tomorrow the main part of the conference starts.  More - later

On big events and big data

The "Big Data" phenomenon gains a lot of traction, interest, and related work in recent years.   The Internet and making everything in digital form has resulted in amounts of data beyond past imagination, and the rate of growth is amazing.   Mark Palmer in his Blog posting made the analog of data as sand, 

saying that  "If every grain of sand in the bucket was 1 byte of data, then:

• The entire work of Shakespeare fills just one bucket of sand (about 5MB)
• A fast financial market data feed (OPRA) fills a beach of sand in 24 hours (about 5TB) 
• Google processes all the sand in the world every week (about 100PB)
• We generate 60% more sand every year" 

Using this analogy - if all data in the world is a sand,  much of the sand is talking about facts, BTW - the fact that a fact appears as a data in the big data universe, does not say that this fact is in fact true.  

Events issue some of this data, but in many cases an event is the fact that a fact becomes true or false, and this fact is not really kept in the data.   

The "Dagstuhl grand challenge",  which is part of the event processing manifesto, is talking about an "event fabric", which will be the Internet equivalent of events instead of data, I guess that the quantities will be on the same cardinality, thus   it will have the same scalability challenge. The main difference is the type of processing -   event processing instead of queries/information retrieval.    Getting to an "event fabric" has indeed many challenges.  In DEBS 2011 there will be a tutorial about this grand challenge.   I'll write more about this challenge in the future. 

(and this is of course Schloss Dagstuhl) 

Event processing and Artificial Intelligence - a tutorial in AAAI 2011

Paul Vincent has recently  blogged  about event driven applications within the applications types cited by the Wired magazine in an article talking about further AI applications.  There are other work within the AI community which look at the short and long term impact of AI research on the universe, such as the AAAI presidential panel on AI research.     There is a lot of potential in collaboration between the AI and EP communities,  from the EP side, there are variety of AI techniques that can be used, and from the AI perspective it provides cases in which AI techniques can be specialized in order to be used in practice.      

Recently my colleague Yagil Engel , an AI person we recruited  to investigate applications of  AI techniques for EP,  and myself have been notified that our tutorial proposal for AAAI 2011, a top AI conference, was accepted.  


So according to the plan we'll be talking four hours in August about the relationships between EP and AI, the first two hours are planned to provide a crash course on event processing for the AI audience, while the second half is planned to discuss various EP challenges that relate to various AI areas (planning, learning, uncertainty handling, autonomous systems and more.    The idea is to establish a dialog with more people in the AI community and pursue this line of research further.    The down side: we actually need to prepare now a four hours tutorial.   Will write more about it when it is finished (I am sharing all my public presentations on slideshare).    

On VLDB 2010 and the event processing tutorial

Still in Singapore,  after the vacation, this is VLDB time,  VLDB is one of the largest databases conferences, however databases is a large heterogeneous area, and I wonder if there is a single person who can understand all  talks.   The keynote talk by Divesh Saristava from AT&T Research talked about stream warehouses, or event stores in my language.  There were also some event processing related demos, and a stream research track, with one interesting talk that compared the semantics of Coral8 CCL language to this of Streambase and got to the (not surprising) conclusion that their semantics is different and similar queries would yield different results, then it tried to come up with a framework to generalize the two types of semantics, and they extend it to other languages.   I think that this is in line of the work we are doing on common model, and will follow up with them (they are from ETH Zurich) about collaboration on that one.


Today I have delivered a tutorial under the title "event processing - past, present and future",  much of it follows the EPIA book.    Since this is a database conference I opened in showing various opinions about the relations between event processing and data stream management, which is the name used in the database community, the various opinions are:

1.  They are aliases -- a stream is just a collection of events, likewise, an event is just a member in a stream, and the functionality is the same.
2. Stream management is a subset of event processing -- there are different ways to do event processing, streams is one of them
3. Event processing is a subset of stream management -- event streams is just one type of stream, but there are voice stream, video stream and more streams
4. Event processing and stream management are distinct and there is no overlapping between them.

As I have heard all four opinions about it,  I'll let you judge which is the right one. Hint:  option 4 is totally false, there is some truth in options 1-3, depending on the viewpoint.


Anyway - the tutorial has been uploaded to slideshare, and you can view it there. Enjoy.


Tomorrow is my last planned day in Singapore, and I'll write more about this very impressive country soon.

DEBS 2010 --- tutorial days and our context related tutorial

This is King's College in Cambridge, a palace shaped building in which I am staying now, and in which the DEBS 2010 conference is being held. This nice picture was taken by Jacklyn Banergee and can be found is a website called Victorian Web. Cambridge is a very nice place and I took some time both yesterday evening and this evening to walk around. There was one thing in DEBS 2010 that reminded of DEBS 2008 in Rome -- it is July, the weather is quite hot, and this place is not very strong on air-conditioning (there is none in the guest rooms!)-- it seems that it is not an important part in the European culture.

The first day of the DEBS conference was yesterday - a PhD workshop, I have attended part of it, today was the tutorial day, I have attended two tutorials - the one about wireless sensor networks (the first half) and the second about event processing reference architectures - you can view the tutorial page of the DEBS 2010 conference for more details. We also had a tutorial that was delivered by Ella Rabinovich and myself on context related stuff. I have written about contexts several times in this Blog, and the tutorial provides some details about the roles of tutorial, in-depth look about tutorials, and implementation of tutorials in practice. I've uploaded the tutorial on slideshare, and you can view it from there, I hope that the rest of the tutorials will follow soon. Tomorrow - the paper presentation part is starting.

On DEBS 2010 tutorials day

My role in the organization of DEBS 2010 was to handle the tutorials day. Starting in DEBS 2008 in Rome, the tutorials day became (for me) the most interesting day of the conference. Today the tutorial webpage is up and running (this is a first version, some missing details will be added soon). Some of the tutorials have been submitted in push mode by their authors, and some have been solicited in pull mode by me.

Six tutorials are planned: three tutorials in parallel in the morning, and three tutorials in parallel in the afternoon.

There will be two tutorials provided by EPTS work-groups, this time:

• A tutorial about event processing architectures given by Adrian Paschke and Paul Vincent who lead this group
• A tutorial by the new work group that deals with the value of event processing to customers, this tutorial will be given by a collection of people. The workgroup leaders are Rainer von Ammon, Guy Sharon and Nenad Stojanovic.

The additional four tutorials - two of them can be thought as extensions to the event processing languages tutorial that Adrian Paschke, Jon Riecke and myself have presented in DEBS 2009 (and was viewed or downloaded by 3551 persons, last time I checked):

• A tutorial about SQL-based event processing, given by Bernhard Seeger.
• A tutorial about Logic based representation and reasoning of event recognition, given by Alexander Artikis et al.

The additional two planned tutorials are:

• A tutorial about event processing in wireless sensor networks that was delayed from last year since the presenter Antonio Lureiro had to cancel last minute
• A tutorial that was prepared with the help of our team here in IBM Haifa Research Lab that will deal with context aware computing and its utilization in event-based systems. The tutorial will be given by Ella Rabinovich (who will be very active in the conference as she will present two papers in additional to the tutorial) and myself. Additional of our team members that helped in preparing the material are: Yonit Magid, Inna Skarbovsky and Nir Zolotorevsky. I'll write more about this specific tutorial closer to the DEBS conference.

The DEBS research and industry tracks program chairs have also sent notifications about rejection and acceptance this week. I think that there are 21 papers accepted (16 in the research track and 5 in the industry track), the research track acceptance rate was 25%.

Our lab is very active in this conference (various people on various aspects), among the 21 papers there will be 5 papers that with all or some authors from IBM HRL, and it will also have co-authors on 2 of the 6 tutorials mentioned above. This is an indication to the focus that the event-based systems receive in the lab.

Reminder: DEBS 2010 and tutorials

The deadline submission for DEBS 2010 is approaching, tomorrow the abstracts for the industry and research tracks should be submitted. My role in this year's DEBS organization is to serve as tutorial chair. Thus, I am using this opportunity to solicit proposals for tutorials. The tutorial program in DEBS started in 2008 and continued in 2009, in both cases there was a tutorial day preceding the main conference, whose content you can find using the links above. The idea of a tutorial is to provide in-depth survey of teaching a topic that is of interest to the conference's audience. Tutorials can be 1.5 hours or 3 hours long. More details can be found in the call for tutorials. Tutorial submission type is next week, anybody who can contribute a good tutorial is encouraged to send. Last year I have participated in giving a tutorial about event processing languages, we have put the slides on slideshare, checking today it had so far 2861 downloads, so it also may have high visibility.

DEBS 2010 site is up

The DEBS 2010 conference site is up now. DEBS is ACM conference, and the major research conference in the event processing area. DEBS 2010 is done in cooperation with EPTS, and will take place in Cambridge, UK. I am in the organizing committee again, this time in the role of tutorial chair, so I'll be soliciting tutorial submissions. The call for papers both in the research track and the industry track, tutorials, demos and more is on the conference's site.

On the DEBS Event Processing Languages Tutorial

In the picture you can see manuscripts in some languages that require some expertise to parse...
Today I spent some time working on my part in the DEBS tutorial day entitled - Event Processing Language Tutorial, on behalf of the language analysis workgroup of EPTS. For this occasion we have also also launched an EPTS presentation logo, designed by Tammy Dekel, the graphical designer of my lab who also designed the EPTS logo.




The presentation starts with the following observation



and will include -- introduction including the language dimensions identified by the EPTS Language analysis workgroups, in-depth description of: SQL based languages, rule based languages, and EPA based languages, and then some advanced topics like: temporal properties, development environments and logic based semantics. The presentation is being prepared in a team work, and also be presented jointly by some of the members. We shall post the final version publicly, after the conference -- we need to finish it first.

There will be some more interesting tutorials in DEBS 2009 including the tutorial of the EPTS use case work group, a tutorial about event processing in capital markets, a tutorial about event processing in sensor networks and more. I am planned to be in Nashville for the DEBS conference, two weeks from now, and will report on the conference in this Blog. Stay tuned.