More on decision model and event processing

I have written before about the "Decision Model" book, when Larry Goldberg visited us in HRL. 
I met Larry again this week in the OMG meeting, and also had dinner with him and Michael Grohs from KPI  in Cary, NC 
(I am now staying nearby in Durham -- almost packed to go start the long journey back).


From their testimony it seems that the decision model is catching quite fast, and hundreds of organizations are using it already in one way or another.  In the OMG meeting they brought their flagship customer Freddie Mac representative, who reported a big success,  there were already several vendors implementing it to be executable.   The benefit is the simplicity and riding on the "spreadsheet" table-like thinking with their methodology.    They are now working on extending the model which started with partial coverage but is evolving,  e.g. they are adding concepts of views and contexts.


Our challenge in event processing is larger, since the complexity of operations is such that I am not sure can be easily expressed in tables, but the seek for abstractions that will enable business analysts and semi-technical users to construct systems is still there.     This is one of our activity areas, which I hope to report progress at some time.   More - later.

On the Virtual event processing symposium: the EPTS use case report

Yesterday we held the EPTS virtual symposium - a kind of live webinar that was broadcast from the OMG Technical meeting in Arlington, Virginia.   This was a complex logistic operation - some of the talks were live on the site, some were live on the phone, and some recorded.  Some obstacles, but it generally worked. There were 77 remote participants and few in the room.    This is behind me and I can breathe some air  - now on a one day visit in the IBM site in RTP, North Carolina.    The EPTS meeting consisted of presentation of the event processing manifesto, that I've described before,  some discussion on standards that I'll Blog about soon, and some report of EPTS working groups.    
The first of them that has been presented was the report of the use case working group, with a first public summary of the survey results.  Pedro Bizarro posted the presentation on slideshare.
While the results reflect those who answered the survey and is not necessarily an accurate sampling of the market situation there are several interesting observations:

1. Event processing applications can be found now in multiple industries (not just financial), some of the applications were in energy and utilities, defense and aerospace, Transportation and logistics, Healthcare, manufacturing and more.
2. The most cited reason for using EP is enhance/improve user services - and this probably relates to adding functionality that was difficult to add other way, others are - cost reduction, agility (faster development) and compliance
3. Many of the applications use database/file as sources, which is more of the traditional way, other are using streaming sources like messages or subscriptions.
4. Most applications are doing various types of notifications -- only a quarter are triggering automatic actions (this is somehow related to the decision oriented observations of James Taylor that I've blogged about recently).
5. Most applications do not require high throughput ( less than 1000 events per second), around 5% of the application need more than 100K events per second -- this is an observation made before by Roy Schulte.
6. Most applications have 100 event producers or less (actually almost half have 10 event producers or less)
7. Among the non-functional properties: high availability was the biggest requirement, while security is still not a big concern 

As written - this relates just to a sample of applications and is not necessarily represent the entire market, but I think that these observations show some of the current trends.  I'll write about other sessions in the webinar later.

Decisions in smarter systems

Arlington., Virginia,  Hayatt Hotel

I am here for the OMG technical meeting.      I have participated in (part of) the decision modeling day organized by Paul Vincent and Christian De Sainte Marie.  Their ultimate goal is to get to a standard on decision modeling, and they have issued proposal for RFP on that issue.  A good survey of that day can be found in James Taylor's Blog.  I sat near James, and he is blogging in real-time.    James himself gave an interesting keynote on the 
importance of decisions  


James concentrated on operational decisions and said in many of the organizations the role of computerized systems is to provide data (in various ways) to manual decision makers when they ask for it.    The smarter systems have larger portion of automated decisions, they are active rather than passive - determine when a decision is needed, and the decisions are measurable with quantitative metrics, so they can be evaluated.  


While James did not talk explicitly about event processing,  it is obvious that it has a significant role in his vision, it has several roles:

1.  determine when a decision is needed
2.  the automated decision itself is often context dependent and  the context can be determined by event processing context mechanism (temporal, spatial, event history related...)
3. the decision itself may depend on event pattern
4. Last but not least -- the extension of event processing to proactive computing coupling with the metric that measures the decision's result can trigger decision to mitigate undesired predicted deviation from the result,(I discussed this one with James during the reception in the evening).  

The EPTS virtual symposium - tomorrow.   A lot of logistics to get it running! 

EPTS virtual symposium - March 24, 2011 registration details

Next week,  Wednesday, March 24, 2011 we are holding the EPTS virtual symposium, broadcast live from Arlington Virgina, in cooperation with the OMG event processing community of practice.  Anybody can attend virtually through the web -- registration to the symposium can be done using this link
The agenda can be found on the OMG site. 

Five years to the first event processing symposium

Today we can mark five years to the first event processing symposium.  In the picture you can see from right to left - Roy Shculte, David Luckham and myself - all of us five years younger.  The fourth partner to organize this meeting was Mark Palmer, at that time from Progress, now in Streambase.    This was the first meeting of the community - before the meeting we did not know each other;  two months before the meeting the mood was quite pessimistic, since it seemed that we'll not have critical mass, as some of those we invited did not think that they are doing event processing, they thought that they are doing stream processing.  Roy Schulte was instrumental to convince everybody that they should participate, we thought of a meeting of 50-60 people and ended up with 72 people.  
The list of participants is here:

Num	First Name	Last Name	Affiliation
1	Tom	Abraham	Xterprise
2	Asaf	Adi	IBM
3	Rohit	Agarwal	Digital Harbor
4	Rajan	Ahmad	Red Rabbit Sotware
5	Tim	Bass	TIBCO
6	Jerry	Baulier	Aleri Labs
7	Steve	Benfield	Agentis
8	Mikael	Berndtsson	University of Skovde
9	Robert	Berry	IBM
10	Richard	Brooks	ISO New England
11	David	Cameron	AptSoft
12	Mani	Chandy	Cal Tech
13	Henry	Chang	IBM
14	Norman	Cohen	IBM
15	Shlomi	Cohen	Actimize
16	Josiah	Collens	MITRE
17	Sanjay	Dalal	Streametics
18	Opher	Etzion	IBM
19	Tom	Fox	Wachovia
20	Bill	Gassman	Gartner
21	Dieter	Gawlick	Oracle
22	Rohn	Griggs	Wachovia
23	Helge	Hess	IDS-Scheer
24	Bill	Hobbib	StreamBase
25	Daniel	Jobst	CITT
26	Chris	Kaprielian	MITRE
27	Paul	Kim	MITRE
28	Harpal	Kochar	Oracle
29	Vaikom	Krishnan	Celequest
30	Chung-Sheng	Li	IBM
31	Tao	Lin	SAP
32	Eugene	Litvinov	ISO New England
33	Ling	Liu	Georgia Tech
34	Roger	Loeb	The MarTech Group
35	Anthony	Lopresti	Systar
36	Noah (Ed)	Loy	MITRE
37	David	Luckham	Stanford University
38	Alan	Lundberg	TIBCO
39	Susan	Malaika	IBM
40	Wolfgang	May	University of Goettingen
41	Shailendra	Mishra	Oracle
42	In K	Mun	MIT
43	Yefim	Natis	Gartner
43	Ido	Ophir	Actimize
45	Elan	Oren	Leanway
46	Mark	Palmer	Progress
47	Francis	Parr	IBM
48	Boaz	Peer	Actimize
49	Ronen	Perlmuter	Leanway
50	Nelson	Pertacek	TIBCO
51	Beth	Plale	Indiana University
52	Greg	Porpora	IBM
53	Arnold	Reinhold	Hurwitz
54	Harvey	Reed	MITRE
55	Jonathan	Rosenoer	IBM
56	Gunther	Rothermel	SAP
57	Aleksey	Sanin	Coral8
58	Roy	Schulte	Gartner
59	Karsten	Schwan	Georgia Tech
60	Marco	Seirio	RuleCore
61	Ed	Shea	FRS
62	John	Trigg	Progress
63	Mark	Tsimelzon	Coral8
64	G	Venkat	Red Rabbit Software
65	Hari	Verma	G4
66	Rainer	Von Ammon	University of Regensburg
67	Elvira	Wallis	SAP
68	Eric	Wayne	IBM
69	Robert	Wilson	Mitre
70	Jeff	Wooton	Aleri Labs
71	Jim	Zafrani	Synthean
72	Stan	Zdonik	Streambase

As you can see we have representatives of various vendors,  some of them do not exist any more,  some customers,  some academic people and some analysts.     

After some search, I found that David Luckham's site also had a page about it which is still there.  

The meeting started with several presentation about different view points on event processing, starting with David Luckham, and ended with Mani Chandy,  who played the devil's advocate.    Some of the speakers of the first day are still involved in the event processing universe like - Mike Palmer, Stan Zdonik who represented Streambase, and now back in Brown University, and the program chair of DEBS 2011;  some of them are not -- like Mark Tsimelzon, who was the brain behind Coral8, and now works for some VC, and Tim Bass, a colorful person represented TIBCO, who ran an opinionated blog for a while, both are not active in the community anymore.

The second day was the application days -  there were groups in parallel that discussed applications of EP in real-time (military) applications, BAM, BPM,  risk and compliance and RFID based applications.  It was interesting to note that the person who took upon himself to organize an algorithmic trading session notified us that due to the "top secret" nature of this application type, nobody agreed to present.  

The immediate action item was to start working on glossary, as we realized that different people are saying the same things in different terms.   I think this has been improved, not totally resolved.

The other consequence (eventually) was the establishment of the EPTS, and the event processing community, which is still there.     talking about the event processing community -  the DEBS conference submissions were closed yesterday, and there is a significant growth in number of submissions (statistics will be published after the review process will end), so the community is still growing.

Five years have passed - and many things happened since then,  but this will always be the first event of the community. .