Back to the Holocaust day, my father and King Alfred

I typically don't publish the same post twice, but today, due to the holocaust day, I am returning to something I published in this blog 3 years ago (and got many reactions upon it), the reason is that according to the statistics, during these 3 years the readers population for this Blog has grown significantly - so while old readers might remember it, I also want to share it with the newer ones,  This day is very much associated  in  my mind with my later father who was the only survivor of a big family.   Here is a picture of my father late in his life: 

I don't have any picture of him until after the  war, but here is a picture of the kitchen the  

Lodz Ghetto, where he worked at the beginning of the war (he said that he might be the person in the middle, but was not sure about it)

My father survived the war, while his parents and 7 brothers and sisters did not.

He never talked much about this period in life,  saying that this was on another planet, and cannot be described.  Once he told me that immediately after the war a relative found him in the survivors list and send him a letter asking him to describe whatever occurred to him during the war, my father answer was:

I will not tell you about what happened to me in the war, instead I will tell you the story of King Alfred  

King Alfred has escaped due to some revolt and was hiding in a farm, when the rebelling soldiers looking for him got there, the farmer hid the King below a big pile of straw; the soldiers started to look at the straw, they nearly removed all of it, and then decided to move on. At later time the King succeeded to overcome the mutiny and returned to his throne. At some point the farmer came to visit him and he said to him -- you save my life, I can give you whatever you wish, the farmer said: I am a modest person, don't need anything, have one question to you, what did you feel when the soldiers almost got all the straw removed. The king has shown an angry face and said: this is a very rude question, hang this man immediately. The farmer was about to be hanged, a rope was already tightened to his throat, and then the king said: stop, now you see what I felt.

On temporal extension to SQL:2011

I have written before about the recent return to the bi-temporal databases, in conjunction with DB2.  In the 1990-ies was the first attempt to create bi-temporal extensions to SQL, at that time there was a language war, some of it is reflected in the book that I have co-edited, published in 1998.  Now after some attempts, SQL:2011 does include support in bi-temporal databases.  The terminology was changed from the original terms.  What was called in the original version - "valid time" is called in the SQL version "application time',  an what was called in the original version - "transaction time" is called in the SQL version "system time".   
More details about the SQL extension can be found in the overview presentation that Craig Baumunk uploaded to slideshare.      As I have written before, temporal database is vital for maintaining historical events, and thus the importance of this standard, and the supporting databases to event processing application is noticable

On lack of monitoring

Recent financial news indicate that the Financial Industry Regulatory Authority has fined one of the analysts firms for failing to supervise equity research analyst communications with traders and clients and for failing to adequately monitor trading in advance of published research changes to detect and prevent possible information breaches by its research analysts.   This is interesting, since it seems that the regulator now expects firms to monitor the consequences of their actions.   This calls for real-time monitoring, monitor causalities between various events, and eliminate increased trading based on unpublished information.     Recently, I have participated in a meeting with stock exchange people in a certain country,  and heard about their efforts to detect undesired phenomena in trade.  They also were looking at real-time monitoring, and even proactive behavior, trying to detect undesired phenomena before they happen.   I guess that we'll see more of these applications from different sides - regulators, traders, and in this case analysts.   

On event server as the 21st century application server

Paul Vincent has posted in the TIBCO Blog a post entitled:  "event server as the 21st app. server".   

Paul cites TIBCO CEO Vivek Ranadivé in TIBCO's quarterly earning report, and concludes that an event server will is a requirements in many applications that process events in various ways.   

Getting to the notion of application server (see illustration below taken from an article on Websphere Application Server)

Application servers are intended to support services to applications such as:  transaction, storage, database approach, security, high availability, administration and more.     

In the event-driven world there are flowing events, and with the Internet of Things, most data in the universe will be in form of events.   In the event processing manifesto (that Paul has been one of the contributing members to its creation) we talked about "event fabric" which will enable Internet scale sharing of events and will support many applications. Some of the fabric properties mentioned were providing services of  privacy, security, interoperability among fabric instances, provenance, energy efficiency, autonomic computing support (self-tuning etc...), availability, scalability, anonymity, non-repudiation, QoS with multiple criteria.     These are some of the services, and there are of course functional services like context service, adapter and transformation service, filtering service, aggregation service, pattern matching service and more that should be built into the server and can be used by various applications from various application areas and types (BPM, CRM, Social computing, track and trace and many more).    

Paul rightly notes that standards have key role in establishing such event server,  Paul indeed wrote the standards chapter in the manifesto. 

I think that the equivalent of app server based on events is inevitable since events will be at the heart of all applications that take sensor data an input.     Work on standards in this area is an old dream, and hope that we'll be able to advance towards it.   

On two-tier analytics

Will Cappel from Gartner has written about two-tier analytics and went back to Immanuel Kant (in the picture above) as support to his thesis.   Kant argued that the human cognition work in two levels:  the first level that grasps objects and raw facts about them, the second level which captures causality between these objects over space and time, applying some levels of simplification to what Kant said, he is right.  Cappel makes the analogy to the analytics world, and says that the first level is satisfied by event processing that process events by filtering, transformation and pattern detection to identify higher level situations.  The second level is satisfied by pattern discovery engines that work on top of the first level.    This is an interesting observation, I think that the picture is somewhat more complicated as there are more tiers.    Event processing detect patterns in real-time, and indeed one of the ways to obtain these patterns are the pattern discovery mechanism over historical data, which may include the results of event processing systems, but should also include many other data items that describe the impact on the environment,  since situation detection triggers actions, and actions impact the environment,  the pattern discovery needs feedback from the outcome along with feedback from the process itself.    The interesting part comes when we add real-time adaptation to the picture, here, in a similar thing to how the cognition works,  the causality relations may change on the fly.   Consider traffic management systems, studies show that these systems are chaotic in nature and one cannot forecast patterns of behavior based on past experience with sufficient accuracy, forecast is limited to about 15 minutes to the future in some cases and the control policies for highway should constantly adapt.    Here we need four tier analytics:

The first tier is the off-line tier which change the setting of the system based on historical learning.  

The second tier is the event processing tier which observes and monitors

The third tier is the real-time forecasting tier which adapts the causalities and make the short term forecasting

The fourth tier is the real-time decision making tier which makes the best decision possible within the time frame allocated for the decision (which may not be the global optimized solution).

Bottom line:  I agree with Cappel about the multi-tier approach,  and pointing out that reality is somewhat more complicated...   

My recent talk in ETH

I used a spoiler of the movie "Source Code" as the opening slide to my recent talk in ETH.   The presentation now is uploaded on slideshare.   Enjoy! 

On smart computing

 I realized that I have not posted in this Blog for two weeks,  I have not really disappeared;  I am involved in a proposal for EU project, and last Tuesday we had the "hearings", which is the second phase of decision about  who will win --- the competition is very tough, and we invested a lot in preparation - in fact, never in my life I spent so much time and energy in preparing to a one hour meetings,  as for the details -- if we'll win this project I'll write more about it.  One detail:  the meeting took place in Luxembourg, where many houses look like palaces (see picture above). 


For now -  catching up with some stuff that was published in the last couple of weeks,  I came across a  Forresters' article about smart computing.    Chris Mines from Forrester writes that smart computing is the next big things and viewing the planned investments of organizations, as seen in the figure below, many of the planned investments are in this area:

The claim is that smart is the next big thing - and mentions three points (copied from the posting)

• Improving transactional processes is yesterday's story. The back-office challenges of preparing financial statements, fulfilling customer orders, or tracking inventory are well addressed by enterprise and personal productivity software. These traditional workloads are migrating to cloud computing resources in some cases, but are not creating incremental technology investments nor opportunities to transform how a business operates.
• Optimizing assets is the next important challenge. Especially in service-based industries, favorable economics result from building an asset base and then selling access to those assets in the form of services. Managing and optimizing the use of those assets -- physical, human, financial, and intangible -- is the source of revenue growth and improved margins. Smart computing systems provide the awareness, analytics, and actions required to improve the utilization of a firm's assets. And in most cases, these systems represent incremental investments in computing resource, with the payback coming in the form of more efficient, more highly utilized, and therefore more profitable assets.
• Smart flexes to meet industry-specific challenges. Each industry has a different mix of assets with different optimization opportunities. In high-tech, development and channel resources must be marshaled in the face of accelerating development cycles, long supply chains, and unpredictable consumer tastes. In finance, mountains of data must be sifted to find the best investment opportunities, while managing market and compliance risk and keeping employees productive. Finding the right mix of online and brick-and-mortar, the right product portfolio, and lean inventories are paramount for retail execs. The right portfolio of smart computing solutions will vary widely across these and other industries

This is in line with my previous posting about the new world vs. the old world, we are in the new world, and the IT industry is gradually shifting towards this new world.     Optimizing assets (traffic, power, human) is certainly part of the game,  eliminating problems before they happen (proactive computing) is another aspect.