Saturday, November 26, 2011

Orson Scott Card's empire

I have been sick and stayed home for a few days,  which gave me an opportunity to disconnect from the professional work and read "empire" and its sequel "hidden empire", both by Orson Scott Card, one of the my favorite writers, whose books are always thought provoking.  The story is around the 2nd USA civil war, between right and left,  where an history professor from Stanford, who dreams about American empire, following the step of the Roman empire, and concludes that like the Roman republic, the American democracy should be eliminated to achieve the empire, thus he orchestrates chaos, civil war, and some other actions, and takes over as an agreed upon president that both democrats and republicans nominate to restore order, later in the second book he takes advantage of a crisis to reshape some of the world and take another step in the empire vision.    There are some questions behind the plot - whether the goal justified the means, since many of the book's hero characters, lose their life as a result of some of the actions,   whether democracy is indeed a value (and if it is - whether today's political system is really democratic).  I think that the clash between the ancient Plato's outlook that the individual exists to serve the society  (Orson Scott Card's president is modeled after Plato's philosopher king as he writes in his epilogue to the books), or Aristotle's outlook that puts the individual in the middle and make society as means to serve the individual.
My own observation is that we gradually move from Plato's universe to Aristotle's universe, and that the current young generation puts the individual in the center.    Societies of all types (including high-tech companies) will require to adjust to this new world.    I have already written about the distinction between these two Greek philosophers, and will revisit this topic again. 

Thursday, November 24, 2011

More on big data and event processing

Philip Howard, one of the analysts who follows the event processing areas for many years, recently wrote about "CEP and big data".    emphasizing the synergy of data mining techniques on big data as a basis to create real-time scoring based on predictive model created by data mining techniques, his inspiration for writing this piece was reviewing the Red Lambda product.   It is certainly true that creation of event processing patterns off-line using mining techniques and then tracking this event patterns on-line using event processing is a valid combination, although the transfer from the data mining part to the event processing part typically requires some more work (in most cases also involves some manual work).    In general getting a model built in one technology to be used by another technology is not smooth, and require more work.  
The synergy between big data and event processing has more patterns of use -- as  big data in many cases is manifested in streaming data that has to be analyzed in real-time,  Philip mentions Infosphere Streams, which is the IBM platform to manage high throughput streaming data.   The data mining on transient data as a source for on-line event processing, and the real-time processing of high throughput streaming data, are orthogonal topics that relate to two different dimensions of big data, my posting about the four Vs summarizes those dimensions.