Showing posts with label event processing courses. Show all posts
Showing posts with label event processing courses. Show all posts

Monday, October 14, 2013

Start of a new semseter and upcoming talks

I have not written recently -- some due to a short vacation (in Paris), and some due to seasonal laziness.. 
In any event - this week the semester started in Israel, and I am giving an Event Processing course again at the Technion, this time I am doing some change in the course.   The course is still based on the EPIA book,  but I am updating now to teach event processing through a model-based approach, which corresponds to the project we are working on recently.  We are going to do first public exposure of this work in ER 2013 in Hong-Kong.  I'll write about it later.  
Next week, I am giving keynote talk in an event oriented workshop co-located with ACM Multimedia 2013.  in which I'll talk about the semantics and modeling of situations and contexts.   Both talks will become public after they are given.   

Tuesday, May 7, 2013

Event processing academic course at the university of Potsdam

I am following academic courses on event processing, and today came across a graduate seminar entitled "event processing technology"  given by Mathias Weske, who is known for his work on business process management,  given in the Hasso Plattner Institute at the University of Potsdam.

It is interesting to note the topics covered in this seminar: 
  1. Scalability: complex event processing solutions for high performance and low latency 
  2. Aggregation concepts: event processing approaches to extract business information from raw events
  3. Correlation: combining BPM and CEP
  4. Uncertainty: handling of noise in data streams
  5. Prediction: predict future events
  6. Heterogeneity: Processing heterogeneous events

All of them are active research topics in event processing.    Some of them are citing our work on uncertainty and proactive event processing.   It will be interesting to collect information about event processing academic courses worldwide and lesson learned from them.  Academic courses are enablers of making a technology part of main stream computing.   


Sunday, February 12, 2012

Crash course to build simple EP application using Esper


A crash course claimed to take less than an hour entitled "A simple introduction to complex event processing" has been posted.  This is done by example, which seems to be indeed very simple, finding "decreasing" or "increasing" pattern over two consecutive events and setting the color as green or red.   The main emphasis is on the setting - how to obtain, define and use events, and configure the engine - threadpools, listeners etc...  However not much about what event processing actually can do -- this is probably the next lesson.


  Esper is contrasted with commercial products since its open source model allows developers to play with it, use it for toy examples, and for daily usage that is not necessarily a commercial application of big enterprise, in our days of enterprise computing this approach has certainly a role to play, it should be noted that Esper is not the only open source in this area, and that some of the commercial products allow free development version (not access to the source code, but enabling developers to use the product for these purposes for free).


Anyway -- if you wish to learn Esper, it is a good start. 

On revision and compensation in event processing

The event processing course that I taught in the Technion has ended (except for the make-up exam in March). This year the students' project has been different from the last couple of years (well, I am bored having the same type of project all over again).  The students were asked to select a research project.  My Teaching Assistant was very skeptic about the students' abilities to do such projects, but after the presentations they did in the last class summarizing what they did so far she changed her mind.    I'll write about these projects after they'll be submitted (due in early March).   One project is investigating the issue of compensation and revision in event processing,  I have written about revisions before,  both in general,  and specifically related to event processing.  
There are couple of motivations of why I have returned to be interested in it now. 


One of them is the investigation of uncertain events,  since more information may be acquired with time about events that are uncertain,  a revision might be needed.
The second is the work on future events,  since future events are obtained using a forecasting process, this forecasting process is not a one-time process, but can be sensitive to additional events that happen between the forecast and the occurrence of the future event, thus the forecast itself may be revised.  
The issue of revision may entail the need for compensation for decisions and actions already taken.

  • In some cases it is easy, example when no action was taken and it is still possible to take action
  • In some cases it is impossible, if an action has been taken, and this action cannot be retracted, 
  • In the remaining of the cases, it might be possible, however not always cost-effective, since it might have cascading effect of compensating for large amount of actions. 


After getting the students' work, I'll write again about this issue. 


Monday, June 27, 2011

A course based on the EPIA book


I got a request today to share the slides that I am using for the event processing course at the Technion,  based on the EPIA book, in order to use it for an event processing academic course.    At this point I will share it in a controlled way for those who wish to use it to give a course,  if you are interested to give a course based on this book, please let me know and I'll give you access to the course's site. 

Sunday, May 1, 2011

Event processing and Erlang



These two pictures are taken from the recent family trip to Barcelona.  In the second one you can see also my wife and my daughters Adi and Daphna (my daughter Hadas is the photographer).  Daphna also uploaded many pictures of the trip to Facebook  (with tagging in Hebrew). 

Recently I came across a posting in Streambase's  Blog,  talking about embedding Erlang in the Streambase development environment, under the title: Streambase loves Erlang.   It is worth mentioning that there have been some views about the use of Erlang in event processing before,  first by Marco Sierio in the Rulecore Blog, a while ago, and a related presentation talking about the use of Erlang in inference rules, was presented by SAP Research.  
Erlang is actually quite an old  language, a functional language that is aimed at massively concurrent applications.   Personally as an old-timer, I like functional programming more than object-oriented imperative languages,  and happy to see a revival of such languages.   It might be interesting idea to write event processing functionality using Erlang.  I am toying with the idea to play with it by means of students projects' next time I am teaching event processing course (in the fall).   I may ask for people who have done it for best practices.     








Tuesday, October 19, 2010

Teaching an event processing course again

Yesterday I have started teaching again, this time in the Bloomfield building at the Technion, seen in this picture.   Sometimes people ask me why do I continue teaching,  it is probably not because I am bored, on the contrary,  I am quite busy;  it is probably not because it is a well-paid job,  actually they pay peanuts.  
I think that I continue to do it, since I like the interaction with students, and it is also means to evangelize "event processing" to the younger generation;  it also provides exposure to graduate students to work with, as over the years I have supervised (or co-supervised)  6 PhD dissertations and 21 MSc thesis, if my count is right.   It is also source for recruiting.   Quite a lot of my colleagues in IBM have been my own students (which I supervised), or my teaching assistants, or students that took courses with me.    


The way my courses are designed revolve among the following principles:


For historical reasons the course is called "database engineering" and always there is a student who expects to see classical database course,  for these students I answer that today in any database conference, data stream management is one of the emerging areas, and I am giving a flavor of it (my approach is not database centric,thought), I also show students some analysts testimonies that event processing is an emerging markets in general.   The second principle is deep dive -- the course is going more on depth then on breadth, the students should gain deep understanding in what event processing is. The last principle is retain over time, my own experience is that to retain over time students have to experience themselves, so they'll get implementation project, I have not yet decided what the projects will be, gave myself time until next week.
I am using the EPIA book as a text book,  but this time will not focus on the FFD example.   
I might create teaching materials for this course so that others will be able to reuse it.


  

Tuesday, August 17, 2010

Some lessons learned from students' projects

Spending some time in looking at the projects of the event processing course I have taught in the spring semester. I am going to teach again this course in the winter semester, starting in late October, the first course that will be based on the final version of the EPIA book.

First - it seems that understanding the model based on the seven building blocks - was quite easy for the students that none of them had any previous knowledge of event processing. It seems that the students could study the model in a relatively short time and represent an application using this model. Of course, getting to the next level of detail in contexts and patterns require some learning, but based on some internal courses we made, I believe that with three days course that includes hands-on lab, one can study the model.

However, the students task was not only to model an application but also to implement it, and they were given the freedom to chose how. Some of the students have used existing products and some of them implemented the application in Java. I guess that here is a micro-cosmos of life, and the world is partitioned among people who like to learn software products and use them, and people who prefer to do their own development.

I have asked the group of students who preferred to write their own why they chose to do it, and their answer was that they are very experienced Java programmers, and while it was quite a substantial project to write it in Java, they felt that the project is "in control" and they are not dependent on any third party (i.e. ability to learn and use a product, possible bug handling etc...) but stay on familiar ground; I guess that this type of consideration also reflects what happens in real life. There is indeed an overhead to study a product for a single project, but some teams thoughts it is cost-effective and succeeded to implement it using a product.

It seems that both type of teams thought that modeling the application first using the model has made their life much easier when they came to implementation since it imposed order. We'll try to have some more modeling efforts of event processing applications using this model to gain more experience (and evolve the model). There are already some ideas and activities around extending the model, but I'll leave it for other time.

Monday, June 21, 2010

On students' projects in the event processing course

The semester has come to its end, and the event processing course in the Technion had the last meeting today; the students still has to finish the projects, and they have some time to do it, but today they have presented a preview of the project, here is a brief summary. They were given a choice to do research oriented project or implementation oriented project, those who chose implementation oriented project got some directions on the size (quantities of consumers, producers, patterns, aggregations, filters etc...), and could chose both the application they want to work on (and they'll have to test it with simulated data) and the tool they use in order to implement.

There are five teams:

  • Team no. 1 -- chose to work on a research problem of interval-oriented event processing and defined a collection of operators and patterns required to support interval oriented.

The rest of the teams chose implementation projects:

  • Team no. 2 -- chose to work on "smart traffic control", simulating traffic lights, traffic jams, accidents and events like concerts etc... They decided to hard-code their application in Java, since they know Java well, and it is more fun doing it this way :-).
  • Team no. 3 - chose to work on "smart home" and mainly concentrate on the security aspect of home automation, they are also Java fans, but chose to use Esper in conjunction with Java.
  • Team no. 4 -- chose to work on control of people who are abusing chat rooms in various ways (like using dirty words), and managing complains. They have chose to work in Streambase, and done all implementation (so far) using the graphical tool of Streambase. We again see that some people like graphical programming and some people prefer old-style imperative programming.
  • Team no. 5 -- chose to work on tracking bus driver performance, both for safety issues (speed), and accuracy (skipping bus stations). They have organized late, so have not started the implementation phase yet, but they said that they wish to do the project using Apama.

The projects are due in August, and then I'll write more, if I'll have any interesting insights.

Tuesday, April 27, 2010

Revisiting Web 2.0 and event processing


To those who like interesting numbers -- I've noticed this week that the number of my contacts in LinkedIn reached 777, it will probably not stay long, as it keeps changing. I have written before about the use of event processing in social networks, an area that I think has big potential that has not been fully explored yet. I have challenged the students in my event processing class at the Technion to select a project that is somehow related to the use of event processing in Web 2.0, and social networks in particular.
I have written before that unlike the phrase, event processing is not always about threats and opportunities. In fact, typically in Web 2.0 the use of event processing is in information dissemination - getting the right information to the right person at the right granularity in the right time. This is a case of situation awareness while the situation here is the fact that some website content has been changed, or that some activity related to social network, Blog, Wiki, and other collaborative applications happened. This can have any of the event processing functions --- filtering seems to be quite dominant, but also transformation - aggregation, translations, splitting all have a role, and of course pattern matching may also have a significant role -- example: send me update after three changes, patterns related to number of readers in a Blog (Google Analytics provides some), patterns related to activities in social networks over time and space, patterns related to tags in twitter and more... I think we'll hear much more about this topic.

Tuesday, March 9, 2010

On new event processing course and some ways to explain what event processing is

This is the Computer Science building at the Technion; yesterday I have started to teach an event processing course there (in the previous semester I have taught in the Information Systems Engineering program, which is my academic home for many years), the course is given as an "advanced topics in computer science" course, and the students that have shown up were mostly graduate students from CS as well of EE (there are software people in EE also). Unlike the previous semester in which the students projects evolved around validating the solutions that different vendors put for the EPIA book's website, this time I would like to concentrate in the project level about the view of those building event processing platform (this is a slightly different view). The first class is always an introduction, since the students don't have a clue about what event processing is, thus I am starting with some examples about what is it used for.
Always there is somebody who asks the question -- what is new, haven't such applications done in the past also? My answer is, consistent with the answer I am giving to this question for years, event processing does not bring any new functionality to the table, it takes functions that people have done in one or other way using regular programming and make it both -- easier to use since it introduces high level abstractions (analog to the abstraction of database query against the way we did database programming in the distant past, and other abstractions), and providing platforms that can execute these abstractions using optimizations specific to these abstractions.
I am also talking about the event-driven decoupling programming vs. the more traditional request/response.

One thing that I am making sure that the students will understand is that (like databases) event processing can be used for many different purposes and is not bound to a single type of application, or single industry. I am doing it by starting with 10 examples that are quite distinct (taken from chapter 1 of the EPIA book), and concluding by trying to generalize some type of applications, using the picture below (taken from an IBM academy study a few years ago that examined what event processing applications are), this is just one of possible classifications.
This is an important thing, since there are some misconceptions around.
As I have written in a previous posting, event processing does not have a single purpose.
Creating event aggregations to check key performance indicators is certainly an application, but not the only one; likewise "detecting threats and opportunities in the event cloud" is a phrase going around (I heard it first from Roy Schulte, but I don't know if he is the copyrighter), and indeed, this is an application of event processing, but many event processing application detect neither threats or opportunities, but serve for diagnosis, or operational decisions. The message - event processing is a discipline with set of concepts, and various ways to implement these concepts that serve many purposes that need event-based computing. I Shall write more about the course as it will further develop.

Wednesday, July 1, 2009

On EPIA and hands-on experience

Yesterday, Peter Niblett and myself, the co-authors of the EPIA book, had a meeting with the publisher and editor in order to summarize the review process of the first 1/3 of the book. One of the topics discussed was the ability of readers to have "hands-0n" experience.



As I explained in a previous posting, our approach is to teach our view about what is event processing and not teaching a specific language. We spent much of the review around this issue, the publisher suggested that for the benefits of readers who would like to have hands-on experience while reading the book we should provide a way to do it. What was agreed is that we'll use the use case that accompanies the book - the fast flower delivery --- to be a bridge between the book and the hands on experience. Since we would like to expose the reader to the multiple approaches that exists in the event processing area -- we'll provide the reader several alternatives to experiment, each alternative will be a language from different type -- it will be emphasized that the languages are just representatives, and the book does not endorse any of them in particular.

For each of the languages we shall provide:

  • The fast flower delivery example coded in this language.
  • Link to a site from which the reader will be able to download an implementation of this language (e.g. trial version of the product implementing this language) on which the reader will be able to run the example and experience in using that language.
Of course, this should be available to the readers without payment. In the next few days we shall talk with some of the language owners to select those who will be able to support these requirements.

I don't know how many readers will indeed spend time to take advantage of this option, but since 2 reviewers (out of 14) mentioned their wish to have "hands-on experience", I guess that there is certain segmentof the potential readers who will do.

Actually, I am a big fan of learning by hands on experience, my own experience from my long years as student is that courses in which I had substantial programming assignments are the course in which the knowledge gained in these courses is retained over time. I'll tell about my experience as a teacher on this issue another time. Stay tuned, this is going to be fun !