On speed and accuracy in event processing

This scary picture is taken from Theo Priestley's post in "Business Intelligence".   As a follow-up to his previous post about the two recent acquisitions in event processing, he talks about the focus in this world on speed.   While speed can provide relative advantage it can also be a double edge sword, if it comes on the expense of accuracy, as the recent Twitter hoax indicates.  

When talking about the the four Vs of big data -  one of them is velocity, and the other is veracity - which is defined as "data in doubt".  Indeed  processing  uncertain, inexact or inaccurate events or data is a major part of what big data is all about -- while there are some works in this area (for example: see my post from last year), it is still the less investigated part of the four Vs.  

Priestley is right -- doing things fast and inaccurate can incur big damage, doing things slow and accurate can also incur big damage.  The wisdom is to balance and minimize the risk.   Resolving the uncertainty issue is the key.   

Maturity model for event processing

This slide is taken from the tutorial that Jeff Adkins and myself are planned to deliver exactly a week from today in Arlington, Texas - DEBS 2013.    It describes our view about "maturity model of event processing - IT view", showing the status of event processing use in an organization.  This follows the CMMI models.  

This model is used to evaluate the current status and determine next steps.  I guess that most organizations are in  levels 1 and 2.       This is first draft -- will be  glad to get comments. 

A proactive system: NASA grand challenge -- mitigating asteroid risks

NASA issued the next in its series of grand challenges, this time the ultimate killer (or saver) application for the idea of proactive systems -  the challenge consists of the sense part -- sensing all moving objects in space,  forecast part --- forecast the asteroid trajectories, determine if the trajectory might jeopardize earth, and if yes -- use robotic vehicles to alter the trajectory.   An ambitious project, which may be vital for the survival of earth.    This includes breakthroughs  in sensors and actuators and  also lightweight robots that can explore asteroids.  Note that there are 8800 known near earth asteroids.  See also the Wikipedia value about "Asteroid impact avoidance".  The relevant research community will have a lot of resources to work on these issues.

Event processing platforms - reboot?

Doug Henschen, the editor of Information Week wrote a commentary entitled "big data reboots real-time analysis" .  Henschen says that event processing was in the height of its hype in 2008, but the economic crisis stopped the growth of this area.  He sees indications of "reboot" in the recent acquisitions of Apama by Software AG and Streambase by TIBCO, and attributes the reboot to the need of big data to evolve from its batch origins to detect patterns on moving data.  

As I have written before, the barriers to growth stem from some external factors (certainly the general financial situation), but also the over-hype of request-response or batch oriented analytics (see my post on Sethu Raman's keynote in DEBS 2012).  Another reason, as observed by Roy Schulte last year,   is that many enterprises developed in-house solutions.    I assume that Henschen is right in the sense that big data gives additional opportunities to event processing technology, and that the recent acquisitions will create waves of interest in the market.   As I have written before, the next frontier is not improving the technology, but making it accessible to the business users and convert the enterprises to think in an event-driven way.   Jeff Adkins and myself will discuss this issue in the coming DEBS'13 tutorial, on June 30.  More - later. 

Web services triggered by SAP ESP

More from the event processing vendors Blogs.  This time from SAP.  Its ESP product.   SAP event processing Blog notifies on a new feature in the product that enables a derived event to trigger WSDL that prompts the user for requesting a predefined web service.  In the EPN terminology, this WSDL is an event consumer.  There are also other products that cross the borders and provide actions as part of the event processing product.  As integration with adjacent systems is a major issue, there are already several solutions which extends event processing to the consumer side, with built-in actions. It is also useful to extend to the producer side -- the instrumentation process.  I'll write about the producer side in another post.    

Is philosophy dead?

I recently followed some discussion threads that started a couple of years ago with Steven Hawkings' assertion: "Philosophy is dead", Hawkings asserts that the truth about the universe is in the hand of the physicists, and philosopher don't keep up with science.   Well, reading it I thought of taking out my BA in Philosophy diploma and hide it, since with the death of philosophy this diploma becomes worthless (maybe I should also retroactive loose my graduate studies, since the acceptance was based on that diploma!).    Some philosophy professors make  defensive claims restricting the death to certain subsets of philosophy.  

My view is that philosophy deals with meta issues,  it does not deal with scientific experiments, but in trying to understand what is a valid argument about the world, and how conclusions are derived from assumptions and assertions.  Many of the concepts we use in computer science: class, object, event, instance and many more actually exist in philosophy much earlier than the invention of computer science.  The recent discussions about causality vs. correlation has long time roots in philosophy literature. 

During my academic studies I have taken courses in various departments: computer science, business school, education, mathematics, but as I have written several years ago, I value the studies in philosophy most.  I think that the study in philosophy has trained  me in both creative thinking and out-of-the-box thinking. I realized early in the studies that this is what you are expected to do in order to get really good grades.   

So I'll trust Hawkings and his colleagues when they talk about physics, and proudly live the philosophy diploma on my virtual wall.  

More on the acquisition of Apama by Software AG

One of the interesting questions about the acquisition of Apama by Software AG is what is the strategy of Software AG going forward in the event processing area, given that it has already acquired in the past an event processing technology from RTM which is named "WebMethods Business Events".

An article in COMPUTERWORLD attempts to shed light on this  issue, citing Stephen Ried from Forrester: "Apama and WebMethods Business Events complement each other; While the former RTM is really lightweight and can be embedded in many Software AG products to provide basic event communication capabilities, the Apama product is for those customers who like a dedicated business event management platform.".

According to this - there are two major use patterns.  Event processing as components embedded inside other products,  I have written before about the component approach to event processing, and indeed not every product needs all the event processing capabilities.  On the other hand, a full fledged event processing application require an event processing platform, optimized for performance metrics.