Tuesday, June 2, 2009

On the methodic use case used in the EPIA book

The EPIA (Event Processing in Action) book that Peter Niblett and myself are writing went to its first milestone -- the first 1/3 of the book (five chapters) have been completed in a draft form (chapter 4 and 5 should still be posted on the web by the publisher), and sent to a set of reviewers that the publisher has selected. One of the main issues in the book is, of course, to make things concrete by using a concrete example. We chose to use one example that accompanies the entire book and is developed step-by-step during the book chapters. One of the good advices we received from one of the initial reviewers on the book outline, was to use an example that does not require any domain knowledge (e.g. trade example), since this can issue a communication barrier with readers not familiar with that domain. Taking this advice, we decided to go for a methodic use case, that takes things from various applications we are familiar this and wraps them up in a single story, which does not require any prior domain knowledge - and this is the "Fast Flower Delivery" use case that I present below.

We present this use case in the descriptive language that we build througout the book, a sample of a building block in this language describing the event structure has been presented in past posting. The intention is also to go beyond that and add (as appendix?) some code samples from languages of various kinds -- SQL extension, rule language, script language etc... some language owners have already agreed to help, and we'll solicit more help from them soon.

Here is the current draft of the "Fast Flower Delivery" use case. It is fairly simple to understand, yet, it includes many (not all) the concepts we explain in the book:

General description

The flower stores association in a large city has established an agreement with local independent van drivers to deliver flowers from the city’s flower stores to their destinations. When a store gets a flower delivery order it creates a request which is broadcast to relevant drivers within a certain distance from the store, with the time for pick up (typically now) and the required delivery time if it is an urgent delivery. A driver is then assigned and the customer is notified that a delivery has been scheduled. The driver picks up the delivery and delivers it, and the person receiving the flowers confirms the delivery time by signing for it on the driver's mobile device. The system maintains a ranking of each individual driver based on his or her ability to deliver flowers on time. Each store has a profile that can include a constraint on the ranking of its drivers, for example a store can require its drivers to have a ranking greater than 10. The profile also indicates whether the store wants the system to assign drivers automatically, or whether it wants to receive several applications and then make its own choice.

Skeleton Specification

Phase 1: Bid Phase

The communication between the store and the person who makes the order is outside the scope of the system, so as far as we are concerned a delivery’s life-cycle starts when a store places a Delivery Request event into the system. The system enriches the Delivery Request event by adding to it the minimum ranking that the store is prepared to accept (each store has different level of tolerance for service quality). Each van is equipped with a GPS modem which periodically transmits a GPS Location event. The system translates these events, which contain raw latitude and longitude values, into events which indicate which region of the city the driver is currently in. When it receives a Delivery Request event the system matches it to its list of drivers. A filter is applied to this list to select only those authorized drivers who satisfy the ranking requirements and who are currently in nearby regions. A Bid Request event is then broadcast to all drivers that pass this filter.

Phase 2: Assignment phase

A driver responds to the Bid Request by sending a Delivery Bid event designating his or her current location and committing to a pick up time. Two minutes after the broadcast the system starts the assignment process. This is either an automatic or a manual process, depending on the store’s preference. If the process is manual the system collects the Delivery Bid events that match the original Bid Request and sends the five highest-ranked of these to the store. If the process is manual, the store makes the assignment and creates an Assignment event that is sent to the system; if the process is automatic then the first bidder among the selected drivers wins the bid, and the Assignment event is created by the processing system. The pickup time and delivery time are set and the Assignment is sent to the driver.

There are also some alerts associated with this process: If there are no bidders an alert is sent both to the store and to the system manager; if the store has not performed its manual assignment within one minute of receiving its Delivery Bid events then both the store and system manager receive an alert.

Phase 3: Delivery process

When the driver arrives to pick up the flowers from the store, the store sends a Pick Up Confirmation event; when the driver delivers the flowers, the person receiving them confirms by signing the driver's mobile device, and this generates a Delivery Confirmation event. Both Pick-Up Confirmation and Delivery Confirmation events have time-stamps associated with them, and this allows the system to generate alert events. A Pick-Up Alert is generated if a Pick-Up Confirmation was not reported within five minutes of the committed pick up time. A Delivery Alert is generated if a Delivery Confirmation was not reported within ten minutes of the committed delivery time.

Phase 4: Ranking Evaluation

The system performs an evaluation of each driver’s ranking every time that that driver completes 20 deliveries. If the driver did not have any Delivery Alerts during that period then the system generates a Ranking Increase if the driver has had more than five delivery alerts during that time then the system generates a Ranking Decrease to reduce the ranking by one point. If the system generates a Ranking Increase for a driver whose previous evaluation had been a Ranking Decrease then it generates an Improvement Note. event indicating that the driver’s ranking has increased by one point. Conversely

Phase 5: Activity Monitoring

The system aggregates assignment and other events and counts the number of assignments per day for each driver for each day on which the driver has been active. Once a month the system creates reports on drivers' performance, assessing the drivers according to the following criteria:

  • A permanent weak driver is a driver with fewer than five assignments on all the days on which the driver has been active.
  • An idle driver is a driver with at least one day of activity which had no assignments.
  • A consistent weak driver is a driver, whose daily assignments are at least two standard deviations lower than the average assignment per driver on each day in question.
  • A consistent strong driver is a driver, whose daily assignments are at least two standard deviations higher than the average assignment per driver on each day in question.
  • An improving driver is a driver whose assignments increase or stay the same day by day

More about the building blocks used in the book will be discussed in future postings.


Hans said...

The last paragraph of your post appears to me in a blue dingbats font and is unreadable.

Opher Etzion said...

Thanks Hans. I am using Firefox for editing the Blog, and did not find any problem, so I swithced to Internet Explorer, and noticed that the entire posting is quite messy, so used IE to revise this time. Lesson: cut and paste from MS-Word is not that simple..



Dress Guard said...

I'm trying to figure out what's new about CEP. Looking at your case study I can't find anything that couldn't be done right now with a classical database system with EAI.

Opher Etzion said...

Dress Guard. This is the same type of question --- why do you need DBMS system and declarative queries, when you can do everything using file systems and imperative programming. You can also using Turing machine to program everything, so the issue is not that it cannot be done otherwise, but the level of abstraction. When the book is out you'll be able to see the solution in EP abstractions, and compare it to a solution based on classical databases system with EAI, and observe the difference.