On the Proasense project

I have not written for a while... Busy days.  I'll write about my daily work in other opportunity.

Last week I've spent a couple of days in Athens, Greece, as a member of the advisory board of the Proasense project.  This project deals with proactive computing, which a paradigm that I have been advocating  for several years.  There are now couple of EU projects I know (the other one is Speedd) in the proactive space.  The proactive idea is that problems can be eliminated or mitigated before they happen.   The Proasense project employs two interesting use cases: one in the manufacturing area, and the other in the oil drilling area, an application that we have investigated a couple of years ago.

I am still in the opinion that proactive computing will be a major paradigm in the future, and will follow this project during the next couple of years with interest.   

Iron Dome: A proactive real-time system that saves our lives

This is a picture of rocket interception by a system called "Iron Dome".    We live in Israel which has been suffering in the last week massive attacks of rockets.  The defense answer to these rockets is an amazing technology developed in Israel which demonstrates the principle of proactive event-driven computing in real life.   The system consists of the detection part which identifies that a rocket has been launched, the prediction part that anticipates where the rocket will hit,  a decision part that decides whether it is necessary to intercept the rocket, this is a cost-benefit analysis.  In many cases the rockets hit empty spaces, in that case it is not cost-effective to intercept them, since each activation of this system is quite expensive.  If the decision is that it is necessary to intercept the rocket, the decision is what is the best point of interception, and  a missile is fired accordingly.  This is a decision support systems with human in the loop.    The missile itself is equipped with electro-optic sensors.    During the last few days this technology has proven very effective and eliminated quite a lot of potential damage to the civilian population.   Many people have been skeptic about the feasibility of this system.  but the results are impressive.  Furthermore it is a good demonstration of the proactive event-based paradigm.    

The blog and myself are going on short vacation in Germany.  Will return in two weeks. 

Proactive event processing for intelligent transportation system

I came across a new publication whose citation is: " Yongheng Wang, "A Proactive Complex Event Processing Method for Intelligent Transportation Systems," Lecture Notes on Information Theory, Vol.1, No.3, pp. 109-113, Sept. 2013. doi: 10.12720/lnit.1.3.109-113".   This paper is a follow-up to our work on proactive event-driven computing,  and applies events coming from the Internet of Things towards intelligent transportation system, proactively mitigating traffic congestion.  This work originates in China, who made Internet of Things as its flagship project.   We have looked at similar problem  as one of our use cases for EU project proposal (that did not win the lottery).

  

DEBS 2012 - our presentation on "basic proactive"

I have written before about the proactive event-driven paradigm that we are investigating in the last couple of years.   Today I have presented some new results in this area, the "basic proactive" model which provides end-to-end solution for a restricted type of applications with several characteristics.  The paper co-authors by Yagil Engel, Zohar Feldman and myself is in the proceedings (will probably be in the ACM digital library soon).   The presentation can be found on slideshare.

Since there has been a slight delay in the schedule, the session chair allowed only 2 questions, one by Alex Buchmann who wondered whether it is wise to introduce new buzzword  like proactive, or position it otherwise.  I think that there are pros and cons, but the buzzword is already there and I'll write about it.

Adrian Paschke, who is interested in the formal aspects, remarked that classical logic will not be sufficient here since conclusions are uncertain and can be retracted.  I answered that we'll probably need non-monotonic logic, maybe combined with some kind of quantitative logic of uncertainty (such as probabilistic logic).  If somebody want to take the challenge and work on the formal model -- let  me know.

More -later. 

Towards proactive enterprise intelligence by Gregoris Mentzas

I came across a recent presentation given by Gregoris Mentzis (from NTUA, Greece) entitled "towards proactive enterprise intelligence".  In this presentation Gregoris discusses some research challenges.

The capabilities of proactive enterprise intelligence are defined in slide 21 and seem similar to our definition (I also recognized the pictures).  I'll write more about the two patterns expressed in this slide. 

Reading this presentation is recommended. Enjoy! 

My recent talk in ETH

I used a spoiler of the movie "Source Code" as the opening slide to my recent talk in ETH.   The presentation now is uploaded on slideshare.   Enjoy! 

Timo Elliot's presentation on "Business in the moment - from reactive to proactive"

Timo Elliot from SAP gave a recent talk in the Gartner BI meeting in London entitled "Business in the moment -from reactive to proactive". You can download the presentation from a link in Timo's Blog posting.   In a following post on his Blog,  Timo refers to an FreshDirect explaining the proactive behavior:

“FreshDirect has an operations center that manages its fleet of delivery trucks. In a large metropolitan area like New York, traffic doesn’t always flow predictably. A traditional approach to BI would be to print a report showing the level of on-time deliveries (OTDs) the day before and then ask the transportation department what went wrong for the orders that were delivered late. FreshDirect uses analytics in a more impactful way.”

“The company monitors the delivery rate of every truck and enters that data into the BI system on an ongoing basis. Every hour, it uses the previous hour’s data to predict how many deliveries will be on-time in the next hour. If the predicted OTD rate is below FreshDirect’s target, the company sends out an auxiliary truck or trucks to help make deliveries. The company holds 10 trucks in reserve for just this purpose.”

I'll bring more proactive stories when I'll find out about them...

Intelligent Business Operations - a medical use case

Within the recent year Gartner promotes the term "Intelligent Business Operations" (IBO)  - not to confuse with Business Intelligence (BI).  Roy Schulte from Gartner wrote about "operational IQ".   I am looking now at the concepts and facilities of IBO, in Gartner's view.    One way to study it is by looking on a recent post by Jim Sinur (also from Gartner).  Jim provides a success story in the medical domain, resource allocation in surgeries.   The ingredients of this scenario are:

1.  Simulation-based optimization of scheduling and resource allocation in off-line for all surgeries planned for the next day.
2. Real-time tracking of everything: physicians, nurses, equipment; monitor of procedure duration and status - using sensors, cameras and in Jim's terminology - exploiting the "Internet of Things".
3. Determination of things already going wrong (not according to plan) or expected deviations from plan
4. Re-applying the simulation based optimization (this time online!) and get updated resource allocation plan.

This may be instance of the "detect-forecast-decide-act" pattern we have identified as the basis of proactive computing, although in Jim's scenario it can also be reactive (the deviation from plan already occurred - there is no need to forecast anything).     


I'll write more about the IBO concept and some additional ingredients of it soon.  
Since the term "intelligence" is now back in fashion,  it would be nice to have metrics for the IQ of some operational process like the surgery management.
2. 

On event processing for real-time and non real-time applications

Chris Carlson has written in the Informatica Blog about the fact that there is a growing segment of applications that are using event processing and are not really real-time, and stating the fact that their share in the event processing market is growing.   I have written four years ago in this Blog about the term real-time and the abuse in its daily usage.  When some people (and marketing messages of vendors) are talking about real-time, they mean "very fast", while real-time really means "within time constraints", the time constraint can be micro-second, second, 5 minutes, or 2 hours.    Indeed, the early adopters in event processing, trading applications in capital markets, are based on low latency and fast reaction.   Many other types of applications use event processing for the functionality of filtering-transformation-pattern matching (or continuous queries in the stream oriented programming style), and the non-functional aspects are secondary.   The area which I am working on these days , proactive computing, has some applications in which there are real-time constraints, but typically not in magnitude of micro-second, but in seconds to minutes.  This is the case where there is a forecast for a future problem (e.g. a traffic jam will occur in 5 minutes), there is a time constraint on activating an action (e.g. within 30 second there is a need to change the traffic lights policies to mitigate the traffic jam).  This is a real-time application, but it has to react within 30 second, to impact in 5 minutes.    The interesting thing is that low latency applications may be "best effort" and not have real-time constraints.  Thus - there are low latency applications, real-time applications, those who have both, and those who have none.  Interestingly, event processing applications can be found in all four groups.  

Reactive and proactive as relative terms

This is one of the most famous visual paradoxes, in this picture one can see a young girl or and old woman, some people see only one of them, with some concentration one see both.   This is a kind of relative view, in event processing there are some relative terms, I have written a while ago about the fact that the terms raw and derived events are relative,  a derived event in one sub-system can be raw event of another sub-system.. 
There are other cases of relative views (an entity may be both consumer and producer, for instance).  


I have reminded on relativism while reading an article about the keynote talk of Jeff Shulman (Gartner's manager of the application integration and web services analysts team).  Shulman is talking about cloud, mobile and CEP as the leading trends for application development and integration.   
As a remark to Shulman's keynote, the article bring an interesting response of  Chris Dressler, VP Technology in Cablevision,  he sees a  CEP can be used to find and correct issues before the end user has the need to make a complaining call.
This is an interesting example,  from the service provider's point of view (cable TV company in this case), this is a reactive application, tracking events that already happened and react to them, from the home consumer point of view this might be proactive, since the consumer may not yet felt the impact of the problem, so from the consumer point of view, this is elimination of problem that has not really happened.    More on this distinction - later. 

Proactive thinking as a cultural change

Yesterday I've returned home from my last trip in Europe (last for the next few months I hope, at least no trip is now planned in the horizon).    I spent Thursday in Zurich in a all day meeting with partners in a consortium we have created applying for EU project,  we have reviewed several use cases, and talked with domain experts in several areas who told us that in their domains, proactive solutions simply don't exist.    Reflecting on that I think proactive computing might be enabler for cultural change.   We are very much focused on technology -- what is the architecture, what is missing beyond the state-of-the-practice, what are all the moving parts -- this is of course important -- but let's look at the bigger picture;  the society is governed by policies, regulations, rules...  Likewise, enterprises like to have strict policies to handle each situations, current analytic software serve this culture -- it typically serves those who make the rules, trying to constantly improve the rules by analyzing the past; the result is recommendation to the policy makers to refine the policies.  
Proactive thinking is different,  here the aim is to determine when following the current rules will yield wrong result and change it, if possible, in autonomous way.   If we take a simple example,   in the Zurich meeting we have discussed traffic management systems -- there has been a lot of work in this area,   there are system who sense traffic congestion and alert about it; there are systems which learn traffic patterns according to hours a day, day in the week, and other relevant segments, and make optimization for the traffic-light policies.  In many cases, traffic light policies are determined by persons who make additional considerations (a powerful politician that lives in some street -  entering and exiting this street in certain hours get priority), anyway, people generally want the power to make these decisions, and prefer that the system will behave according to consistent rules.  Proactive thinking in this case is identifying anticipatory traffic congestion and change the traffic light policies accordingly, overriding the regular policies, trying to resolve a specific problem.


This is a cultural change, since the culture typically prefers that the measures will be statistical, and the behavior of systems will be consistent,  this preference has deep cultural roots in certain societies, and reflect the inability of people to get real-time decisions, or improvise.    Those who will adopt proactive thinking may go beyond the cultural handicap and achieve relative benefits....  more about it - later

On proton-based chips

Asimov talked about positronic brains for robots, maybe one step was the construction of proton-based chips, instead of the regular electronic-based computing (reference made by Rainer von Ammon on the complexevents forum).  The proton-based chips can have better potential to communicate with biological processes and serve as an actuator within the human body, even the human brain.    When we called our proactive computing project in the name "proton", we did not know about such a chip,  but it can have natural connection:  The proton system makes decisions that are carried out using the proton chip that serves as an actuator for the proton system.  Cool idea -- somewhat futuristic, but we are getting there.   While the scenarios we are working on now are still within the familiar domains,  I believe that the big potential of proactive systems are in robotics and computerized-control of biological processes.   

What's in a title?

Some people noticed that I've changed my title in LinkedIn and sent me queries about it.  I did not change my job today;  as I noticed that my business cards are about to end and I need to issue a new one, I thought that this is also a good time to re-think on the content, and determined that the title I have there today does not really reflect what I am doing, so I switched to a better title, principle investigator of the Proton project, which is part of the IBM Research's  "far reaching research" program (FRR), which we have extended, to explore the emerging paradigm of proactive event-driven computing.  In the past I moved from working on concrete project to a more general technical strategy role, and then coming back to research, in working on requirements and challenges on the next generations of event processing, and towards the end of last year I moved back to concentrate on a single project which I believe has a huge potential.     I have written about proactive computing before,  and we exposed some of the ideas in the DEBS'11 paper.
I'll write more about this project in the future, and of course I am still writing about general event processing issues. 

A video clip from NEXT - demonstrating the proactive idea

The credit for this idea goes to my colleague Zohar Feldman.


A good way to illustrate the proactive idea is to watch the video clip from the movie NEXT,


In this movie Nicolas Cage acts like a person who can see  two minutes into the future, in the clip he is trying different alternatives how to approach a woman he is trying to pursue, understanding why they fail he improves it until success is achieved.     This illustrates the proactive idea:  event happens, their consequences are predicted and a decision what to do in order to achieve some goal should be taken.  There might be different decisions, but we need to predict the consequences of these decisions and make the best one that will get us to the desired solution.     Enjoy!

On temporal databases and DB2

I have written before about temporal databases in this Blog, and in general I worked on temporal databases around 15 years ago, and co-edited the book whose cover is shown here in 1998.  Temporal database is noted by the fact that academic people tried to drive standard in this area: TSQL2 lead by Rick Snodgrass and his colleagues. At that time it did not succeed since none of the DBMS vendors had an interest to see it as a high priority, these were the days were the Internet emerged along with XML, and the DBMS vendors had many other things to worry about.  However over the time  some DBMS vendors have adopted temporal capabilities within their products. Oracle already implemented temporal extensions supporting TSQL in its DBMS product.  Recently IBM produced its own version of temporal database within DB2.  It seems that there is now traction for temporal databases in various industries.    Today my colleague Guy Sharon attracted my attention to a new article on IBM DeveloperWorks entitled "going back in time" describing the DB2 temporal capabilities,  the traditional dimensions in temporal databases: transaction time and valid time got converted to new names:  system time and business time  (I make a note to write a post about the overuse of the term "business" ).     These two dimensions enable to ask queries like: what was the value of a certain attribute in 7/7/2011, as observed from 8/8/2011.  This can have different answer from the observation time of different days, since the knowledge about the past is changed in time.  While the title of that article talks about "going back in time", and indeed using temporal databases is typically viewed about recording the past, temporal databases can also be used about recording the future, this was noted in a work published in 1994 by Arie Segev, Avi Gal and myself entitled "retroactive and proactive database processing"  (I don't think that online version is available).  Since we are dealing over the last year in proactive event-driven processing, the issue of looking at predicted future events that can be revised with time is very useful, and we are indeed looking on temporal database techniques for that.  More on that - later.

On event processing and robotics

In my standard presentation about proactive event-driven computing  (I gave one of those in Almaden earlier this week),  I always show these two pictures:   I am about to spill coffee on my laptop, and my robot detects it and grasp the cup.    This is the simplest example to explain the idea and people understand it and typically smiling (or laughing),  in one instance one commented that it is not cost-effective to use robot for these purposes, which is probably true, but he missed the point of what I've tried to do.
BTW -- spilling coffee is something I do always on a daily basis (today I spilled ice-coffee), I am very talented in spilling coffee on everything (though until now not on a laptop, I have a rule of not drinking the coffee too close to the laptop), so having such a robot would be extremely useful to me!   


In the EPTS meeting in Trento we heard a talk by Sebastian Wrede about event-based robotics,  but I admit that I have never followed up with the robotics guys.    


Today in AAAI there has been a keynote talk by Kurt Konolige about robotics,  after the talk I came forward and introduced myself by saying that the closes I came to robotics was reading Asimov books (which  probably was a bad starting-point for a conversation with him), and explained him a little bit about what event processing is and then moved to the coffee spilling scenario.  His reaction was that in the robotics world they deal which more more basic stuff:   How to improve the vision and perception, and how to calibrate the robot's arm to be more accurate, so he thinks that the speed in which the robot will identify that I am going to spill coffee and the accuracy in which he can grasp the cup are not in the level that the current robotic technology can do this scenario -- which does not mean that I'll stop using it!   His message is that before robots will be able to participate in "high level" tasks, as he called what I described to him, there are many "low level" problems to be solved.      Well - I'll have to wait patiently,  or find some other pointer in the robotic community who will be interested in the synergy between robotics and event processing -- I should definitely start active search.  

Revisiting "Right Time"

This illustration (as indicated in the bottom) is taken from the "enterprise irregulars" site, posted by Ray Wang.
Ray Wang cites a relatively old posting of mine, talking about real-time, right-time and other time related concepts.   I admit that sometimes I abuse the term real-time (like other people do, but this is not a good excuse!), but I have not adopted the term "right-time".   In that posting I bring some classical definitions of various types of real-time.  Wang is making a somewhat different classification as a matrix with two axes:  the reactive/proactive axis, and the business value axis (low/high).  The high value proactive is called "anticipation", and the low value of proactive is called "nice things to do".   My interpretation is that both deal with notifications that may allow proactive behavior, but not necessarily automated proactive behavior of the type that we talk about (see my keynote talk last year in the OMG conference),  on the reactive front, the high value are mission critical reactions. and the low business value are called "timeless responses".  Here, I am not sure it is the best title, as there are reactions that have low value, but are time dependent, since they lose their relevance in time.  Example here is that getting an alert on available discounts in a nearby store may not be that important for me, but the discount is applicable only within the next hour, so if I would like to respond, there is time bound on this response.   Anyway - interesting classification. 

Smart cars -- towards proactive elimination of car accidents

Thanks to a posting of Rainer von Ammon in the CEP forum,  a "killer" application of proactive computing,  is the MIT smart cars that predict human behavior of other drivers and intended to eliminate accidents.  This is intended for the interim period where some of the car will be autonomic and some manually driven.  The rate of success is high, still not perfect.   Actually this is the opposite of a killer application - it is a life saver. 
I guess that in the future, all cars will be driven by computers and this will be able to eliminate the accidents (or minimize them),  since human behavior cannot be totally predictable,  More on ability to predict events - later.   

More on black swan events

Today I had a procedure of tooth extraction and bone transplant.  so I am semi-functioning with stitches in my mouth for the next week.    The reasons for the fuel pollution that paralyzed all flights from Israel yesterday is still in investigation, so the event was not classified yet to black or white swan.  If it will be classified to a black swan event this will be a second instance of the same event within 5 years, so it is interesting if a second instance qualifies as a black swan.  

Some comments about my previous posting claimed that a black swan event cannot be predicted by definition, however event processing can act to decrease the damage and mitigate its outcomes,  maybe the consequences of the black swan events can be predicted and some action need to be taken.  I guess that one of the interesting applications of proactive computing is to deal with consequences of black swan events, and then monitor white swan events to track the progress towards or away from undesired consequences.    More on this topic - later.

Decisions in smarter systems

Arlington., Virginia,  Hayatt Hotel

I am here for the OMG technical meeting.      I have participated in (part of) the decision modeling day organized by Paul Vincent and Christian De Sainte Marie.  Their ultimate goal is to get to a standard on decision modeling, and they have issued proposal for RFP on that issue.  A good survey of that day can be found in James Taylor's Blog.  I sat near James, and he is blogging in real-time.    James himself gave an interesting keynote on the 
importance of decisions  


James concentrated on operational decisions and said in many of the organizations the role of computerized systems is to provide data (in various ways) to manual decision makers when they ask for it.    The smarter systems have larger portion of automated decisions, they are active rather than passive - determine when a decision is needed, and the decisions are measurable with quantitative metrics, so they can be evaluated.  


While James did not talk explicitly about event processing,  it is obvious that it has a significant role in his vision, it has several roles:

1.  determine when a decision is needed
2.  the automated decision itself is often context dependent and  the context can be determined by event processing context mechanism (temporal, spatial, event history related...)
3. the decision itself may depend on event pattern
4. Last but not least -- the extension of event processing to proactive computing coupling with the metric that measures the decision's result can trigger decision to mitigate undesired predicted deviation from the result,(I discussed this one with James during the reception in the evening).  

The EPTS virtual symposium - tomorrow.   A lot of logistics to get it running!