On Positive Thinking

Two news items have been highlighted in the Israeli press, seemingly unrelated: the first one is Obama's speech in Egypt addressing the middle east people, and the second one is about Dudu Topaz, a famous Israeli entertainer, who was arrested by sending hired people to beat quite hard some senior people in the Israeli media. In the speech that the U.S. president has given earlier this week he addressed the people of this region and called upon them to exercise positive thinking and overcome the differences, to get a permanent settlement. I would say that he called upon the people to desert the "zero sum game" and move to "cooperative game". Some people around me say that this message is very naive, and asserting that since the other side has negative attitude then positive attitude is useless. It is always easier to unite people around negative messages, and indeed the current government of Israel has been elected using negative messages that the people through the collective wisdom of democracy decided to endorse. The other side is not innocent either, generation of children are educated on negative messages of hate. I very much support Obama's call for positive thinking, it will not be easy for people to think that way, after a lifetime of negative thinking, but IMHO this should be the way forward.

The other piece of news was somewhat surprising to most, as written, a famous Israeli entertainer, who was in the past the "king of rating" in commercial TV, hosting a popular show (whom I have never watched). A few years ago, after some embarrassing incidents that he was involved in, the commercial TV stations decided not to hire him anymore, a person like him who felt like a king, with a huge ego, could not stand the humiliation and frustration in repeating rejections and hired some bullies to beat some people hard (they needed hospital treatment and surgeries to undo damages) - one of them was CEO of one of the commercial TV stations, and the other was a program manager of another commercial TV station (the fact that she is a woman did not matter to him), the third was the artist's own agent that did not succeed to take care of him, and the police found in his house plans about several other people. This is an extreme type of negative thinking, but it worthwhile writing something about it. It happens to many people that one day they are stepping out of some position, or some circumstances change, and suddenly they are out of their previous power anymore, some people are going on and doing other things, but some stay on the sideways of the court in which they played before, and feel very frustrated by the fact that they are not the players anymore, since they are convinced that they would have played much better. This lead people to invest a lot of energy in negative thinking and negative actions. Dudu Topaz is an extreme case, but I have seen and am seeing various such cases of investing energy on negative thinking.

I can certainly understand frustration, the wheel is spinning, and as many people, I have been some time up and some time down, but early in my life I decided that positive thinking is much better attitude to life, and looking back is not really a good policy, remembering what happened to Lot's wife when she looked back. Although like any human, I am sometimes tempted to have negative reactions, which mostly proven to be the wrong ones.

I have written in the past about positive thinking in Blog posting, but it is true for other activities as well.

In early stage of my life I have read the poem "IF" by Rudyard Kipling (in Hebrew translation) and felt that this is not just written words, this is a code of behavior that I should adopt. I have returned to cite Kipling from time to time, when I am thinking on a way to behave in extreme situation. I have copied Kipling's poem below his picture.


If (R. Kipling)
If you can keep your head when all about you
Are losing theirs and blaming it on you,
If you can trust yourself when all men doubt you
But make allowance for their doubting too,
If you can wait and not be tired by waiting,
Or being lied about, don't deal in lies,
Or being hated, don't give way to hating,
And yet don't look too good, nor talk too wise:

If you can dream--and not make dreams your master,
If you can think--and not make thoughts your aim;
If you can meet with Triumph and Disaster
And treat those two impostors just the same;
If you can bear to hear the truth you've spoken
Twisted by knaves to make a trap for fools,
Or watch the things you gave your life to, broken,
And stoop and build 'em up with worn-out tools:

If you can make one heap of all your winnings
And risk it all on one turn of pitch-and-toss,
And lose, and start again at your beginnings
And never breath a word about your loss;
If you can force your heart and nerve and sinew
To serve your turn long after they are gone,
And so hold on when there is nothing in you
Except the Will which says to them: "Hold on!"

If you can talk with crowds and keep your virtue,
Or walk with kings--nor lose the common touch,
If neither foes nor loving friends can hurt you;
If all men count with you, but none too much,
If you can fill the unforgiving minute
With sixty seconds' worth of distance run,
Yours is the Earth and everything that's in it,
And--which is more--you'll be a Man, my son!

Back to professional postings -- soon.

On temporal semantics of events - or when has the shimpent not arrived ?

In the early 199o-ies, my home away from home, has been Berkeley, where I stayed for a joint work with Arie Segev on temporal databases. In one of the weekends I have strolled along the famous SF Fisherman's wharf, and there was some store for left handed people, since I am part of the deprived minority of left handed people, I was curious and entered the store, among the different items there (mostly not very practical), I saw this clock, if you notice, it is a backwards clock, which goes anti-clockwise. I am sure that the owner of the store was right handed -). Anyway, I recalled this clock, when working on a final version of a paper entitled "Temporal perspectives in event processing" that has been accepted recently for publication, and re-read the paper (as any paper, it is written, submitted, and then after a few months a review arrives and the author has to be reminded what it was, revise according to the comments, send back, and so on, until it is either accepted or rejected), and thought that temporal semantics of events can be a good topic to write about here. The temporal semantics of a backwards clock is, of course, different than that of the regular clock, and this brings me to the temporal semantics of derived events. Some background: event may have two time-stamps (or intervals) associated with it: occurrence time and detection time. Occurrence time is the time that the event happened in reality, detection time is the time in which the event processing system detected the event message sent to it. It is easier to make the processing of the events (when did they happen ? in what order ?) according to the detection time, however, for some applications, this may yield incorrect results. There are several issues around obtaining the correct occurrence time, but let's assume that we know how to do it. While the occurrence time of a raw event (events that has arrived from an external producer that assigns the value) is explicitly provided, the question is what is the occurrence time of derived events. Let's take a simple example: In May 2nd, 10:30 the customer John Galt has issued an order for books, with a guaranteed delivery of 48 hours (see my story with Amazon in its early days as a footnote to this postings). In May4th, 10:30 Mr. Galt looked at his (forward going) watch and said: "the shipment has not arrived by its deadline". The fact that he has not reported on arrival by the deadline caused the event processing system to derive the event "shipment did not arrive", which is a time-out event (or non-event event as some vendors call it). Now the question is WHEN did this event happen ? the detection time is easy, when some computational process derived the event and emitted it to the event processing system then the detection time is set. Let's say that this happened in May 4th in 10:32. The occurrence time is more tricky. Actually I can think of three different interpretations:

1. The occurrence time of the "shipment not arrived" is the same as its detection time, which means May 4th, 10:32.

2. The occurrence time of the "shipment not arrived" is the deadline of when it should have arrived, in this case, May 4th, 10:30

3. The occurrence time of the "shipment not arrived" is the entire interval of the 48 hours, since the shipment did not arrive during this interval [May 2nd 10:30, May 4th 10:30].

What is the right answer for semantics ? there is no right answer, as some more cases in event processing, the system designer should chose among these (and may be other) alternatives.

More about temporal semantics -- later.


Footnote: A story from the early days of Amazon.

I was an early customer of Amazon, buying, science fiction books through the web (I still do it). Typically it took 3 week for a shipment to arrive to Israel, so once after three and half weeks in which the packaged did not arrive, I've sent Email to Amazon customer service to ask about it. Their response was surprising -- we don't know what happened, we are re-sending you the books. After two more days I received the original package, and since I thought that may be the substitute package still can be stopped, I send another Email to Amazon friendly customer service, and got event more amazing response -- after we issue the reservation we cannot control the rest of the process-- so please keep the extra book with our compliments. At that time I thought that this company is not going to survive... Of course, since then they have much better logistic system... and I have two copies of each of the books in this shipment.

On the methodic use case used in the EPIA book

The EPIA (Event Processing in Action) book that Peter Niblett and myself are writing went to its first milestone -- the first 1/3 of the book (five chapters) have been completed in a draft form (chapter 4 and 5 should still be posted on the web by the publisher), and sent to a set of reviewers that the publisher has selected. One of the main issues in the book is, of course, to make things concrete by using a concrete example. We chose to use one example that accompanies the entire book and is developed step-by-step during the book chapters. One of the good advices we received from one of the initial reviewers on the book outline, was to use an example that does not require any domain knowledge (e.g. trade example), since this can issue a communication barrier with readers not familiar with that domain. Taking this advice, we decided to go for a methodic use case, that takes things from various applications we are familiar this and wraps them up in a single story, which does not require any prior domain knowledge - and this is the "Fast Flower Delivery" use case that I present below.

We present this use case in the descriptive language that we build througout the book, a sample of a building block in this language describing the event structure has been presented in past posting. The intention is also to go beyond that and add (as appendix?) some code samples from languages of various kinds -- SQL extension, rule language, script language etc... some language owners have already agreed to help, and we'll solicit more help from them soon.






Here is the current draft of the "Fast Flower Delivery" use case. It is fairly simple to understand, yet, it includes many (not all) the concepts we explain in the book:

General description

The flower stores association in a large city has established an agreement with local independent van drivers to deliver flowers from the city’s flower stores to their destinations. When a store gets a flower delivery order it creates a request which is broadcast to relevant drivers within a certain distance from the store, with the time for pick up (typically now) and the required delivery time if it is an urgent delivery. A driver is then assigned and the customer is notified that a delivery has been scheduled. The driver picks up the delivery and delivers it, and the person receiving the flowers confirms the delivery time by signing for it on the driver's mobile device. The system maintains a ranking of each individual driver based on his or her ability to deliver flowers on time. Each store has a profile that can include a constraint on the ranking of its drivers, for example a store can require its drivers to have a ranking greater than 10. The profile also indicates whether the store wants the system to assign drivers automatically, or whether it wants to receive several applications and then make its own choice.

Skeleton Specification

Phase 1: Bid Phase

The communication between the store and the person who makes the order is outside the scope of the system, so as far as we are concerned a delivery’s life-cycle starts when a store places a Delivery Request event into the system. The system enriches the Delivery Request event by adding to it the minimum ranking that the store is prepared to accept (each store has different level of tolerance for service quality). Each van is equipped with a GPS modem which periodically transmits a GPS Location event. The system translates these events, which contain raw latitude and longitude values, into events which indicate which region of the city the driver is currently in. When it receives a Delivery Request event the system matches it to its list of drivers. A filter is applied to this list to select only those authorized drivers who satisfy the ranking requirements and who are currently in nearby regions. A Bid Request event is then broadcast to all drivers that pass this filter.

Phase 2: Assignment phase

A driver responds to the Bid Request by sending a Delivery Bid event designating his or her current location and committing to a pick up time. Two minutes after the broadcast the system starts the assignment process. This is either an automatic or a manual process, depending on the store’s preference. If the process is manual the system collects the Delivery Bid events that match the original Bid Request and sends the five highest-ranked of these to the store. If the process is manual, the store makes the assignment and creates an Assignment event that is sent to the system; if the process is automatic then the first bidder among the selected drivers wins the bid, and the Assignment event is created by the processing system. The pickup time and delivery time are set and the Assignment is sent to the driver.

There are also some alerts associated with this process: If there are no bidders an alert is sent both to the store and to the system manager; if the store has not performed its manual assignment within one minute of receiving its Delivery Bid events then both the store and system manager receive an alert.

Phase 3: Delivery process

When the driver arrives to pick up the flowers from the store, the store sends a Pick Up Confirmation event; when the driver delivers the flowers, the person receiving them confirms by signing the driver's mobile device, and this generates a Delivery Confirmation event. Both Pick-Up Confirmation and Delivery Confirmation events have time-stamps associated with them, and this allows the system to generate alert events. A Pick-Up Alert is generated if a Pick-Up Confirmation was not reported within five minutes of the committed pick up time. A Delivery Alert is generated if a Delivery Confirmation was not reported within ten minutes of the committed delivery time.

Phase 4: Ranking Evaluation

The system performs an evaluation of each driver’s ranking every time that that driver completes 20 deliveries. If the driver did not have any Delivery Alerts during that period then the system generates a Ranking Increase if the driver has had more than five delivery alerts during that time then the system generates a Ranking Decrease to reduce the ranking by one point. If the system generates a Ranking Increase for a driver whose previous evaluation had been a Ranking Decrease then it generates an Improvement Note. event indicating that the driver’s ranking has increased by one point. Conversely

Phase 5: Activity Monitoring

The system aggregates assignment and other events and counts the number of assignments per day for each driver for each day on which the driver has been active. Once a month the system creates reports on drivers' performance, assessing the drivers according to the following criteria:

• A permanent weak driver is a driver with fewer than five assignments on all the days on which the driver has been active.
• An idle driver is a driver with at least one day of activity which had no assignments.
• A consistent weak driver is a driver, whose daily assignments are at least two standard deviations lower than the average assignment per driver on each day in question.
• A consistent strong driver is a driver, whose daily assignments are at least two standard deviations higher than the average assignment per driver on each day in question.
• An improving driver is a driver whose assignments increase or stay the same day by day

More about the building blocks used in the book will be discussed in future postings.