Event processing in gaming: SAP and Bigpoint

Jeff Wooton from SAP posted on the SAP event processing blog a video clip showing a project done with Bigpoint, which is a gaming company, to get events from players in real-time and make offers to players based on their progress in the game and prediction of patterns of their behavior.   The solution is based on Hana, SAP in-memory technology, and I guess on their event processing technology also, though the video does not give much details about the actual implementation. The goal is to maximize profit by personalize sells to game players.   Event processing has various applications in game playing, as game has event-driven behavior nature, and also in dynamic personalization, which is applicable to other areas as well.

More on fraud detection from IBM - the MoneyGram case

I have recently written about real-time fraud detection solution from IBM based on SPSS and CICS. 

Another solution from IBM in the fraud detection space has been recently highlighted by David Luckham in his site citing a Forbes article.   This solution is based on the "Entity Analytics" solutions (specifically the Infosphere Identity Insights product.    David Luckham comments "that are a lot of CEP under the hood of  this stuff".  This is true.    After the acquisition of SRD by IBM, our team in IBM Haifa Research Lab helped integrating our work on Amit as embedded part in the Entity Analytics -  (see the DEBS 2010 presentation by Ella Rabinovich on Industry Experience with Amit - slide 3).   This integration seems to have survived all these years and being employed for useful purposes. 

In any event,  there are many solutions in which event processing is integrated with other technology to create specific solution, and we can see alliances between event processing technology and other technology in this area.

On Ayn Rand

Ayn Rand had a significant impact on me when I was young.  I read her books while being a teenager, and was captured by her ideas.   Recently, her two famous books "The fountainhead" and "Atlas Shrugged" gained new translations to Hebrew, which was a good opportunity to read them again after many years.  

In retrospect, there was something naive in my attitude to Ayn Rand's book as a young person, there is also a lot of misinterpretations of her ideas in the universe, one of them is the claim that this is a basis for conservative thinking. Ayn Rand was far from being conservative in the political sense.    

However, two things I have taken from Ayn Rand, escorted me all my life:

The ability to "swim against the current"  and take what other people think of me with a grain of salt (including my managers over all  the history),  and the strive for excellence in the mediocre culture around us.  

These two are blended together in Ayn Rand's books. 

 She puts the individual in the middle, rather than the society, and puts in the center of her stories individuals with some kind of excellence. 

Our current society seems to prefer mediocre people that meet metrics, rather than individuals who break out of the box (thus the metrics don't follow them).      A food for thought.

Prelude to the DEBS 2013 tutorial

The notable event of this week in our family was the wedding of our eldest daughter Anat. In the picture you can see Anat, her new husband Adi with my wife and myself, in fancy dresses.

It was also very loaded week at work, one of the items has been submitting extended abstract of the  tutorial that Jeff Adkins and myself are  planned to deliver in DEBS 2013.     So here is the tentative outline we submitted in the original proposal:

Outline  

Topic I:  Introduction – Brief history of event processing  

Topic II:  The major differentiation factors of event-based thinking

Topic III:  The ontology of events and event influence

Topic IV:  Anatomy of reactive systems 

Topic V: Pragmatics – a computational independent model for event-based systems 

The tutorial starts with follows some of my previous postings about "event oriented thinking",   and Jeff's 4D classification.  It analyzes the way that people think about systems in the conventional way vs. the way people think in event-driven fashion, and gets into the role of events in language, and in system modeling.  

Unlike the implementation oriented thinking we usually employ -- this is a computational independent thinking, it looks at the event-based systems from the customer's perspective.     See (at least some of) you in Arlington.

Event processing - small data vs. big data and the Sorites Paradox.

This picture is taken from a blog post from the "Big Data Journal" by Jim Kaskade entitled "Real-time Big Data or Small Data".  

Kaskade attempts to define quantitative metrics to what is "small data" vs. what is "big data".  

In terms of throughput big data is defined as >> 1K event per second, while small data is << 1K per second, I guess that around 1K event per second is defined as medium data...  

On variety big data is defined as at least 6 sources of structured events and at least 6 sources of unstructured events.  There are other dimensions like - small data relates to one function in the organization, while big data to several lines of business.     

The attempt to define where "big data" starts is interesting, the main issue is what are the conditions in which implementation of systems should become different, and here the borders are not that clear, since there are currently systems that can scale both up and down.

Interestingly -- "Big" and "Small" are fuzzy terms.  Which reminds me on one of the variations of the Sorites Paradox,  that I've came across during my Philosophy studies, many years ago, which goes roughly like this.

Claim:  Every heap of stones is a small heap.

Proof by mathenatical induction.

Base:  A heap of 1 stone is a small heap

Inductive step:  Take a small heap of K stones and add 1 stone, surely it will stay a small heap.

Causality vs. correlation - statistical reasoning is not enough - NY Times Interview with Dave Ferrucci

Dave Ferrucci, who was until several months ago an IBM Fellow  and was known as the father of Watson, was interviewed by the NY Times in his new working place at Bridgewater Associates.

In the interview Ferrruci somewhat continues the line of thought of Noam Chomsky,  saying that AI has concentrated around statistical reasoning based on correlations, but the drawback is that one cannot understand why the prediction made by the statistical reasoning is correct.  While Chomsky bluntly stated that statistical reasoning does not create a solid model of the universe, Ferruci claims that a complementary approach is required -  understanding causality.    This is a rather old issue, in symbolic logic, there is a distinction between "material implication"  which states that  IF A is true then B is true, and the meaning is that always when A is true then B is also true, which makes a sentence like  "If the week has seven days than  the capital city of France is Paris" - a valid statement in logic.    Entailment, on the other hand, said that "A ENTAILS B" if it is necessary and relevant, in other word, there is a causality among them.  Thus, Ferruci concentrates now on building causality models to model the world economy.      I concur with the assertion that understanding causalities give better abilities of reasoning and prediction.   As David Luckham already noted, causality among events is one of the major abstraction of event processing models.   Here is a rather old discussion about causality of events.  

Event processing academic course at the university of Potsdam

I am following academic courses on event processing, and today came across a graduate seminar entitled "event processing technology"  given by Mathias Weske, who is known for his work on business process management,  given in the Hasso Plattner Institute at the University of Potsdam.

It is interesting to note the topics covered in this seminar: 

1. Scalability: complex event processing solutions for high performance and low latency 
2. Aggregation concepts: event processing approaches to extract business information from raw events
3. Correlation: combining BPM and CEP
4. Uncertainty: handling of noise in data streams
5. Prediction: predict future events
6. Heterogeneity: Processing heterogeneous events

All of them are active research topics in event processing.    Some of them are citing our work on uncertainty and proactive event processing.   It will be interesting to collect information about event processing academic courses worldwide and lesson learned from them.  Academic courses are enablers of making a technology part of main stream computing.