Monday, December 13, 2010

On Hadoop and event processing


The region which I live in did not have much luck recently, first the big fire on the Carmel ridge, that lasted for three and half days until it got under control, and now a major storm, with winds running in velocity of >  100 KM/H  and a lot of rain.   These two pictures, taken from the Israeli news Internet sites, were taken in Haifa yesterday.   The storm is now over and some nicer days are ahead of us.


Back to professional issues -- Alex Alves (who represents Oracle in the EPTS Steering committee among other things) wrote a nice posting in his blog explaining the Hadoop programming model, if you are still not familiar with it, it provides good explanation. 


Hadoop is batch oriented and provides kind of imperative programming model, but can be wrapped and concealed by higher level language.     I am working now with a graduate student who investigates the usability of the map-reduce model for some of the event processing functions (e.g. aggregation).   I am curious  to see the analysis of this work.   More - later

2 comments:

Hans said...

This student may be interested to google "mapreduce online" if they have not already found the paper of that name.

Richard Tibbetts said...

Another interesting paper is recent work from google on Percolator: http://www.google.com/research/pubs/pub36726.html