Showing posts with label event processing in action. Show all posts
Showing posts with label event processing in action. Show all posts

Sunday, August 15, 2010

First review of the EPIA book on Amazon



Since the book Event Processing in Action became available, I am keep getting Emails from various people about it. The first one who has put a review on Amazon was Jim Odell, a notable person in agent technology, who recently wrote a series of articles about agents and events, hosted in Paul Vincent's TIBCO Blog. The review is entitled: "A true magnum opus". Since Manning publications is doing three reviews during the development process of the book, we got several dozens of reviews throughout the development of the book, many of them included some criticism in various aspects, and responding it required much work, in one case, a major rewrite. Some of the feedback I am getting about the book includes also some ideas for additions -- well, it is still early to think about second edition.

Saturday, April 18, 2009

On Event Processing Building Blocks

Back to work for one day in the office, with five conference calls (one with Germany, one with France, one with UK, and two with USA...) and then back to home for the weekend. When I have free time I like to read books, the current book I am reading is "A Lion Among Men", 0ne of the books of Gregory Macguire, who writes stories that take as background famous children stories (in this case - the Wizard of Oz), actually this is the third one behind the scene of the Wizard, now taking the Lion as its main character. I have another book of the same author still waiting...

We also submitted the draft of chapter 3 of the "Event Processing in Action" book to the publisher, which hopefully be posted on the MEAP site soon.

The approach we have taken in the book, as I have written before, is to use the "building block" approach, describing event processing principles, and the use case whose construction demonstrates the application, using building blocks, which are like the chemical elements. The application itself is being built by using "definition elements" which are like atoms (my partner for writing this book, Peter Niblett, has come with the analogy from the world of chemistry). we believe that this is the right approach to teach what event processing is -- in the "deep dive" part of the book we dedicate a chapter for each of the major seven building blocks and then dive deeper into the types of event processing agents (which deserves a different discussion). We'll also provide samples of how each building blocks is realized in different models.

The seven building blocks are:
  • Event type: defines the event schema
  • Even producer: the projection of the event producer over the event processing network (note that the event producer itself is outside the scope)
  • Event consumer: same -- the projection of the event consumer over the EPN.
  • Event channel: the glue that holds the EPN together
  • Event processing agent: the brain that does the entire work; each agent is doing a specific task of processing.
  • Context: the semantic partition of events and agents
  • Event derivation: A building block that is possibly part of each EPA that specifies the derived event.

There are some more building blocks that are used to support these ones, but our claim is that this set of building block is what needed to build an event processing application.

Chapter 4 which is in advanced phases of being written starts the deep dive by discussing the event type building block, and in one of the next posts I'll say more about it.

Saturday, April 11, 2009

Some footnotes to the forthcoming book "Event Processing in Action" - Take One

Last night, I went to see a movie (a rare event for me) -- and chose to see "Slumdog Millionaire", my daughter told me later that people who have not read the book enjoyed it more, the movie is OK, even cute, however -- for a movie who won 8 academy awards, I have somewhat bigger expectations (comparing for example to "Gone with the wind" who also had 8 academy awards. Well -- the movie industry is probably not peaking these days...


Today, together with (most of) my tribe, we have done some hiking in a place called "Judge River", well, river in the local terms, with a modest amount of water, but bridge, a lot of trees, some flowers, and since it is a holiday, a lot of people.

Now back home and like any Sunday morning I plan to go to one of the coffee shops (I am rotating between the coffee shops in Haifa, well, to be exact, those who have free parking nearby) to work on revisions to the draft of chapter four of the "Event Processing in Action" book.



From time to time I'll blog about giving some footnote from behind the scenes of the book-being-written. Today I'll blog about several issues: scope, language and exercises.

Scope: The idea is to focus about teaching the event processing concepts step-by-step using a use case which will accompany the book throughout, so the question is -- what is the scope of event processing. We define this scope by defining the "event processing network", and thus the question, that I started discussing in my precious posting is -- whether pre-processing and post-processing to the event processing network is part of the event processing network. While we have a chapter that is dedicated to event producers (and pre-processing) and another chapter that is dedicated to event consumers (and post-processing), the scope of what we discuss as part of the specification of the event processing part do not include what is done by the producers and consumers, whose projection on the EPN is the events they produce and consume. However, there is a case in which a consumer is also a producer, and this is important since there is a possible causality relationship between the event it consumes and consumer and event it produces. As an example: the use case is talking about "fast flower delivery" and one of its functions is choosing the driver that will get the delivery among the drivers that has issued a bid. Some of the stored prefer automatic assignment by the system, and some want to get the bids and do the assignment on their own. The automatic assignment is definitely an EPA (Event Processing Action), since this is a software that performs some operation on events, however the manual assignment can be either manual, or the store is using some external software to do it, however, this is not really part of the EPN, thus it is not modelled by the system. We are of course interested to trace the assignment to the bid which is the input to the store. This is also a good example to show that the same event type can include both raw events (the manual assignment are raw events from the EPN POV) and derived events (the automatic assignment).

Language: We decided neither to use any single language to explain the concepts, nor to invent a new language. However, we believe that just a theoretical discussion will not be enough. What we have decided to do is to take a "building block" approach, in which the different parts of the systems (event types, event processing agents etc..) are specified using "definition elements" which are platform independent concepts, or in other words, meta language. In each section we'll provide the full part of the application using this meta language, in order to connect it to the "ground", we'll also make samples of these definitions using variety of languages in various style. Thus, chapter 4 that I am writing now talks about defining the event schema. We define the schema using our "event type" building block, and will also show definitions in various schema languages (XML, positional relational-schema-like etc..), the same will go for all types of event processing agents. We intend to ask owners of existing languages (from those who will agree to get their languages analyzed by the EPTS event processing languages analysis -- taking on another hat) to provide language definition of our use case, and will check the possibility of posting them all.

Last but not least are the exercises, as we want the book to be a textbook for academic course on event processing, as one of its targets, we have decided to put exercises at the end of each chapter for the benefit of the students and instructors (we also plan to provide slides in the future), one of the questions we agreed with the publisher to ask the reviewers (there is a formal review for each 1/3 of the book) is whether this is the right way or it can make other readers uncomfortable. The options are now: leave as is (exercises at the end of each chapter, make all exercises as appendix or remove them completely from the book, and have them available on a website).

That's all for now -- more footnotes - later.

Wednesday, March 25, 2009

Visiting Aston University and MEAP for "event processing in action"


This are pictures from Aston University in Birmingham which I am visiting today within my short trip, it turns out that from the university network I cannot get into the IBM VPN, so my hosts brought one of the university IT guys who confirmed that they are blocking access to private VPNs, as a matter of security policy, and the only person who knows how to configure a bypass is away today, and even if he has not been away, he doubts if he would have willing to do it, I could in principle go to town an look for an Internet Cafe, but I am too lazy, and can live a couple of days without Emails. Besides that the visit (which has not been over yet) has been interesting and we may have grounds for collaboration . I have given my talk about "event processing - the next generation" third time this week (fourth time tomorrow in another place...), I'll post it on the web after my trip. One of the question I was asked was whether (complex?) event processing techniques could have predicted the economic crisis, but I'll leave discussion about predictions to another Blog posting. They also may teach an event processing course next year and are looking for a textbook to base the course upon.

Speaking of book -- the publisher of the "Event Processing in Action" book has gone another step and included the EPIA book (currently the first two chapter drafts) in the MEAP (Manning Early Access Program), the referenced site explains how readers can become part of the authoring process by receiving draft of new chapters, and making comments/questions to the authors using the forum to ask questions and communicate with the authors. As mentioned before the introduction chapter has been posted as a green paper, and is free download to all. Although open type of writing is somewhat more difficult and time-consuming to the authors then just write the book without interruptions, I believe that this process can improve the quality of the book beyond the formal review process. So this is a call for the community to take advantage of this program and help in creating this book. Hans Gilde has already made the first set of comments for Chapter one.

I am not sure whether it is related to blogging about the book being written, but yesterday (Tuesday the 24th of March, 2009) has been a record high in terms of amount of visitors for this Blog in a single day, today has not ended it, and is also looks strong, so I wonder why.

Tomorrow -- visiting the IBM Hursley Lab in UK, before returning home.



Wednesday, March 18, 2009

Event Processing In Action


Event Processing In Action - this is a title of the book on which I have started to work recently together with my colleague Peter Niblett - although I am typically writing my Blogs always as I and not as We, this time I'll use We in any case that what I am writing refers to Peter as well. In the next few days the first chapter of the book will be made public by the publisher as a green paper. In the picture above you can see a provisional cover of this book, but this is not final yet. The book is planned to be available towards the end of 2009.

The Web 2.0 plays a role in this process, as explained below. Here are some Q&A about it,


What is the motivation for a new book ?

The book has been initiated by Manning Publications, a computing books publisher; their market survey indicated that there is a significant market need for a new book that will articulate and provide a deep dive into the concepts and facilities of event system applications. This book is intended to be the major reference book for enterprise architects, application developers (both technical and semi-technical), and is also expected to be used for instructional purposes (a textbook for a university level course on event processing).

The book written by David Luckham entitled "The power of events" (Addison-Wesley, 2002) has been very influential in setting the initial awareness to the event processing area, and it still is a big inspiration for us; the new book is intended to reflect the contemporary thinking around event processing which has been evolved since 2002.


Why have we agreed to write this book ?

Writing a book is a big responsibility, it is a substantial burden on our time. Furthermore it is a tremendous challenge to produce a high quality book in an emerging area for these target audiences - especially considering the very high expectations that have already been generated around this book. We believe that this book is indeed required, and as technical leaders in the community it is our duty to take this task and help shape the newly emerging discipline of event processing this way. We were also encouraged by our management and colleagues to take this mission.

What is the approach taken in this book ?

The approach taken in the book will not be surprising to the readers of this Blog. Indeed, the book can be considered as a direct descendant of the Blog, it seems that the publisher has approached me based on recommendations of anonymous members of the event processing community that referred him to look at this Blog. I got feedback from others that this Blog is one of the popular sources today to learn what event processing is, but the Blog, as a Blog, is not written in methodical way, it jumps from one topic to another, it treats the various topics in a relatively superficial way, and includes "noise" like this posting; the book should be more focused, getting things in the right order, and in the proper level of depth. The style of writing is similar to that of the Blog.

The book will explain all the event processing concepts by showing step-by-step how a single use case has been constructed. The explanation, like my approach in the Blog, is aimed to be language-style neutral and explain the concepts using a patterns oriented model (although, due to the ambiguity of the term patterns in event processing we use the term building blocks). We are planning to have an appendix in which we will list existing EP products and open source offerings, and provide some high-level details, without providing evaluation or endorsement to any of them. We'll ask for collaboration of the various product owners to get accurate information about their products.


What is the relationship between this book and IBM ?

Both Peter and myself are IBM employees; Peter works in the IBM Hursley Lab in England, where I am working in the IBM Haifa Research Lab in Israel. However we are writing this book (after clearing the legal and managerial permission) as individuals and not as IBM employees; A disclaimer stating that the book represent our opinions and not necessarily the opinion of IBM will be clearly made in the preface to the book, as is done in the top of this Blog. There is a big EP oriented community inside IBM and we hope to get feedback from this community, as part of the feedback from the larger community.


How are Web 2.0 technologies going to impact the authoring process?

As any other book, there is a formal review process, in which the publisher consults with a collection of reviewers representing people from the target audiences, thus most reviewers are architects and developments from various industries, and academic instructors teaching EP courses. In addition, nowadays, book authoring is also considered as an interactive process between the authors and the readers. The MEAP program (Manning Early Access Program) enables readers to interact with the authors through a forum, and contribute comments and questions on the book while being written; when the book will get into the MEAP program I'll further explain it


What are the next steps?


As I have said, Peter and myself are facing with a substantial challenge to create a high-quality book for the readers, and are sure that feedback and reviews from the larger community can help us provide a better book for the target audience; The green paper is due to appear hopefully by the end of this week; I'll post the URL on this Blog as soon as it is available, the MEAP for this book will be set up in the next few weeks. I'll also use this Blog to tell about some dilemmas and challenges in the writing process (another Web 2.0 means of communication).

More -- Later.