Internet of Things - what's holding us back?

InformationWeek published an article this week by Chris Murphy entitled: "Internet Of Things: What's Holding Us Back".   In this article Murphy describes several reasons that hold us back from exploiting the potential of the IoT.  The reasons he mentions are:

1. The data is not good enough:  the claim is that the conception that all requested data is readily available is not consistent with reality, where data suffers from quality,  frequency and spatial coverage of the sensors, and data integration issues.
2. Networks aren't ubiquitous:   The product owners don't have control over the availability of networks
3. Integration is tougher than analysis:  The main problem is not to analyze the data, but to integrate all data needed for analysis
4. More sensor innovation needed: The stated areas of required innovation are - combine video sources which today are under-utilized; more-refined and more-affordable environmental sensors; software-defined sensor,a combination of multiple sensors plus computing power that sits out on a network and "calculates rather than measures."
5. Status quo security doesn't cut it.  Security systems for IoT should be radically different than those developed for traditional IP.

I agree that all of these contribute in one way or another to the difficulties around exploiting the potential of IoT.    Dealing with inexact or uncertain data is a major issue, a link to our tutorial on the topic can be obtained from this blog post.  What Murphy refers as "software defined sensor", is in fact, the ability to use multiple sensors and get sense out of it in real-time,  this is exactly what the event processing discipline produces, furthermore, our work on event modeling contributes to make it simpler. 

I am planned to deliver a tutorial on "Internet of Everything" in DEBS 2014 in Mumbai, where I'll discuss all these issues.  

More - later. 

On event semantics -- my talk in the event-based multimedia workshop

Still in Barcelona for the event-based multimedia workshop.   I still need to write on the second day and the closing panel (in which I also participated), and my impression about the event-based multimedia community.

Meanwhile -- I have uploaded my presentation to slideshare, parts of it is reuse of other presentation (e.g. the slides explaining the notion of context),  the new stuff is about the semantics -- who are the players in the event game.    I am planned to give a long tutorial on event modeling in ER 2013 next month - stay tuned. 

On the OODA loop and the 4D

Richard Veryard asked in a comment to my post on  the recent tutorial given by Jeff Adkins and myself if our 4D scheme (which was introduced by Jeff)  is related to the famous OODA loop.  I have mentioned the OODA loop in the past in connection with the need to act faster than the speed of thinking. the strategy of air combats, later he also made claims about the generality of this method.    The 4D is certainly from the same family, and the four stages are indeed similar.   Interestingly Boyd's had event-driven thinking.    

The OODA loop was aimed to describe event-driven decision by a human - the human has to observe that an event happened, perform mental self orientation to analyze the meaning and implication of the event,  decide what to do, and act accordingly, have feedback loop to see whether the observation has changed.

The 4D describes a computational process, where the event is detected (not necessarily directly observed),  situation is derived (by computational means and not by mental process), and then a decision is taken (autonomic or manual) and an action is performed.   

The mapping is not 1-1:    

detect is always mapped to observe;

derive can be mapped to observe - as the detected situation is a derived event, and sometimes to orient - as it may derive a conclusion.

decide can be mapped to the combination of orient and decide in the OODA loop

do seems to be always mapped to act. 

More thoughts about the 4D and related stuff - later.  

Our DEBS 2013 tutorial: why is event-driven thinking different than traditional thinking about computing?

The tutorial that Jeff Adkins and myself delivered in DEBS 2013 yesterday is now posted on slideshare.  

Many of the participants reacted with further discussions.  The first take away we wanted to convey is described on this slide.   While the community continues to work on advancing the technology (which is fine!), the main challenge today is not the technology,  but making the paradigm shift in people's mind about 

event-driven thinking.  Indeed, experience shows that a typical developer when facing with an application which we classify as event-driven would tend to implement it in a traditional way:  treat the events as data, store them in a database, and process them in request/response mode.   The tutorial discusses the differences and explains the ontology of event-based system as we see it.  It also has a short description of our recent work around the "Event Model", on which I'll write more in the future.  

More about DEBS 2013  --- soon. 

On event model and decision model

In the article about "event model" by Luckham and Schulte, on which I have reported last week,   the term "decision model" was referenced as one of the complementary models to an event model.  This has created some discussion within the DMN LinkedIn group, the discussion was created by Paul Vincent who is one of the persons behind the DMN standard work.   I am following the DMN work, though not in an active way, and participated in the kick-off meeting in March 2011,  and also written on "decision models and event processing"  following the talk on "The Decision Model" by KPI at the same event. 

In order to view the relationship between "event model" and "decision model", we need first to understand what is a decision that a decision model is modeling 

 The DMN presentation defines decision as "A Decision is the act of applying decision logic to one or more inputs and produce one output".    This can relate to modeling a collection of business rules, which gets a request with an associated input and returns a result  ("concluded fact") or to a result of some quantitative model (such a scoring model or optimization model). 

In comparison event model is modeling event-driven logic, which is based on deriving situations or derived events based on filtering, transformation, aggregations, and pattern detection over a collection of input events.   

The question is what is the relationship between these two terms?  

we can start by looking at the relationship between business rules and event processing,  there have been long discussions about it in the past, the position I expressed was that related to a discussion with James Taylor in 2008, stated that event processing and decision management are different issues.  A decision may not be related to event, and an event processing may not trigger any decision.  However, there are also relations between them.   

In my recent presentations about event-driven applications I am using the "4 D" that coined by Jeff Adkins: 

 I found out that this is an easy way to explain both IT and business persons who are not familiar with event-based systems what it is in a nutshell.      Referring to this picture, an event model deals with the "detect" and "derive" phases.    The derived event may also by itself denote a decision (since it is obvious what to do if this situation is detected and does not require further decision),  or that the detection of this situation triggers a decision that can be modeled by a decision model (note that the situation may trigger a task, and then the decision may be embedded inside a business process model).  In the example referred to in the slide above,  the  "derive" phase detects that a traffic jam is coming, and the "decide" phased determines how to reset the traffic light policies in order to reduce the traffic jam -- in this case they are complimentary.

Another interesting point is that "decision" in the common interpretation is "request driven",  I have recently written about the distinction between "event driven" and "request driven".   One may claim that a request is an event (which is sometimes semantically doubtful), or that there is always some event that causes a person or a system to make a request (may be metaphysically true, but the event is not explicitly exposed).  However, request-driven is taking an action by request (a passive approach), while event-driven is taking an action without a specific request, due to either occurrence of event or detection of situation (an active approach).    An event can take 

Bottom line:  decision models and event model are complementary models.   I'll discuss similarities and differences in the content of what is modeled in a later phase. 

Event model - what comes first - the event data model or the event driven logic model?

As promised, I'll write some issues related to event modeling, which is my current focus of interest.

The current thinking on event processing and actually on some related areas (such as BRMS), and also reflected in Luckham and Schulte's article on event models is that the first step is to build the data model, and then model the logic on the basis of the data model.     According to this methodology there is a need to model the used data and events, sometimes also modeling "business objects" that maybe created from the original data or events.   When modelling the logic it always relies on the existing events and data already modeled.  This is a "bottom up" approach.    

 A major issue with this methodology is that it moves the modeling immediately to the IT domain, since business users can relate better to goals, processes, and business logic then to data models.   Keeping the modeling in the level of business require to reverse the process,  model the business logic first, and leave the data and events as "place holders" which need to be completed later in the process.  

A top down modeling is also reflected in that article when the phase of "identify the complex events" precedes the phase of "identify base events".   I think that in the business user's terminology it is better to talk about situations that need to be reacted upon then on "complex events" that is more technical (and somewhat ambiguous) term.   Of course, modeling is an iterative process, since when getting back to the base events or data, a gap (e.g. event that is not being observed) might be discovered.   Thus it seems that the methodology should be:  start top-down (goal oriented) and tune up bottom-up.

More on event modeling issues -- later. 

Event model - the next frontier

I am writing this time from Falls Church, Virginia.  Where I arrived yesterday for a business trip in the Washington DC area, and then in Toronto.   So far I was not affected by the weather, but we'll see what happens later in the week.     

The slides above is an old slide which I used seven years ago to show the relationships of events and the rest of the universe.   I was reminded on that when reading a new article by David Luckham and Roy Schulte entitled "Why companies should develop event models?".    In this article they explain the motivation and provide a high level methodology to build event model.     Following what I recently written about "event oriented thinking",  it seems that the major obstacle to adoption of event-driven systems is the ability of people to think in event-driven way and not in the traditional request-driven way.   Being able to create event model and relate it to enterprise computing modeling is an essential step.    This is the direction I am working on these days (very much in the direction that Luckham and Schulte indicate), and will surely write more about it in the sequel.