Sunday, June 29, 2008

On cost-effective EP when high performance is not required

I have already referred to this issue in the past. There are several concrete examples that come to mind - here is one (real system):
An enterprise has internal regulations to handle suppliers, these requirements relate to the interaction with suppliers, that may be represented as events, and involve actions that must be done, or should not be done, within certain amount of time, or until some event happens. The application is to monitor the compliance with these internal regulation in audit mode, meaning the results are alerts (which are also treated as events, since there are time-outs from alert sending) and not direct interference in the business processes.
The throughput is far below what is considered as "high throughput" and it is several thousands events per day. The latency is also not required to be extremely low -- if the alert will be issued within a minute or two, it is still very fast auditing. It also does not require any analytics of intelligent procedures, since the regulations are given and deterministic
What are the benefits for the customer to use EP software and not any other solution?
First - some of the regulations are fairly complex - writing them in hard-code can be time and resource consuming, putting everything on databases and using SQL was also considered - but some of the regulations are not easy to express in SQL.
Second - Regulations tend to change frequently, the users wish to control these changes, and getting them through the slow IT development cycle will delay the introduction of the change.
So in this case the customer's motivation are - agility and de-complexity; more later


Brian Connell said...

Hi Opher,

Good example. To be accurate, I asked for an example where Volume and/or Latency is not a factor...

You say that SQL is not considered because it is too difficult to express. So what are you using that's easier?

Is it not the case that loading all the data into a database and running a query is impractical because it would simply take too long and take too much resources?

So perhaps the real reason for using CEP in this case is that it is more efficient on resources (perhaps spreads a load more effectively across a longer period of time), and reduces storage requirements (throws away data that doesn't get matched in a pattern, no need to keep all the data for each SQL query).

If so, this is still a factor of volume.


Opher Etzion said...

Hi Brian. In this example the volume and latency indeed was not a factor; the factor has been that the functionality required in the application has fit better CEP than other type of languages (e.g SQL). The factor may be stated in economic terms as - Total Cost of Ownership (development + maintenace) cost.