CEP is a lightweight and agile complex event processing (CEP) engine. It aims to ease the cost of changing business logic, automating and monitoring business processes, and enabling an on demand business environment. Our unique CEP capabilities were recognized by Gartner as an emerging market in which "enterprises will achieve new levels of flexibility and a deeper understanding of their business processes by applying the techniques of complex event processing to their daily work".
The flow of Complex Event Processing
CEP extends the message at a time paradigm, by detecting complex situations that involve a context-sensitive (semantic context, temporal context, spacious-temporal context) composition of messages and events. This functionality is extremely practical when applied to the processing of input from multiple event sources -- from the perspective of a business, application or infrastructure within different contexts. For example, such scenarios may involve SLA (Service Level Agreement) alerts and compliance checking, applicable to the financial markets, banking, and insurance industries.
Complex Event Processing can be performed in many different solution architectures, with a number of CEP (EPA) agents.
The following are some sample rules or events that can be monitored and detected using CEP:
- Suspicious money transfers - send notification if the total value of all withdrawals (ATM, check, etc.) in the last X days exceeds $Y (where X and Y are parameters that each customer can specify)
- Fraud detection - send an alert notification if two credit card purchases were made within one hour at a distance greater than 200 Km.
- Market trends - check for situations where the foreign exchange rate is up by more than X percent and Dow Jones is up by less than Y percent as compared to yesterday's closing rate.
- Customer care - send an alert if the same request was reassigned to three agents and no answer was given to the requester.
CEP lets business users and application developers configure the middleware in terms of business needs, thus triggering more sophisticated conclusions and enhancing automation and efficiency. CEP can be configured to monitor situations relating to specific rules/events. For example, you can pre-set and automatically correct parameters for breaches of Service Level Agreements.
CEP technology includes an Eclipse based authoring tool for the business user, and a Java-based execution engine for complex event processing.
- At build time, rules and conditions (called patterns) are composed using a wizard based authoring tool. The user can also use simulation to test these definitions and then export them to the runtime engine.
- During runtime, events of different format and from multiple sources are fed in. The CEP engine analyzes them based on current rules and conditions. Upon detection of the predefined patterns, alerts are sent out as messages and actions are triggered. These messages are also fed back into the engine and treated as incoming events, enabling nested patterns.
The build-time part represents the rule definition phase. During the runtime phase, events enter the system, and they are processed, correlated, and analyzed to detect complex events.
The following diagram presents the concept of the CEP runtime architecture. All inputs to the system, including both the rule definitions and the events, enter the system through the Input Adapters. All the outputs of the system, including situation alerts, definition messages and system messages, go to the Output Adapters, and optionally to the Action Managers, if an action is required, for example, sending an email or updating a database. This flexible architecture allows the CEP engine to be integrated into any system, to receive inputs and then send alerts to continue the natural flow of information in the system.
CEP Runtime Middleware Component Architecture
First, definitions are loaded through the Input Adapters to the Definition Manager. Definitions are parsed, and if they are consistent and complete, they are loaded into the Rule Engine through the Routing Manager. Success / Fail messages are sent to the Output listeners. At this point events can begin to flow into the system, upon their occurrence. Events enter from multiple sources through multiple Input Adapters. The events are routed to the Rule Engine, in order to participate in evaluation of a rule. When a rule is satisfied, a situation is detected.
A detection of a situation may result in three outcomes: Detected situations are sent to the Output Listeners, for further processing of the information; Situations may trigger stand-alone actions, such as sending email, storing information in a DB, etc.; Situations return to the system as incoming events, thus allowing nested rules to be executed.
Haifa's CEP rule engine is a unique attempt to build general purpose event processing engine. CEP is based on previous research focused on specific fields such as active database and network management. We extended pervious concepts and developed a flexible event processing rule engine while resolving several research challenges such as:
- How to support different kind of events and messages
- How to synchronize events that arrive from different source in different time due to network delays.
- How to identify the context (temporal, spatial, semantic) in which a situation detection is relevant
- How to change event processing rules without stopping the system (hot updates)
- How to support vast numbers of events and conditions in an optimal way
- How to detect cycles in rule firing sequence.
For the past seven years, the CEP team in the Haifa Research Lab has been working on complex event processing technology and event driven solutions, from defining and refining the CEP definition language and implementing and optimizating the run-time execution engine for high volume even processing in distributed environments, to building business level CEP tools and integrating them into IBM products, solutions and customer projects. This work on event processing was published in dozens of papers and patents and is helping to form an event processing community.
Asaf Ad, David Botzer and Opher Etzion. The Situation Manager Component of Amit - Active Middleware Technology. NGITS. 2002.
A. Adi and O. Etzion. Amit - the situation manager. VLDB Journal, 2004.
Asaf Adi and Opher Etzion. The situation manager rule language. RuleML. 2002.
Asaf Adi, Opher Etzion, Dagan Gilat and Guy Sharon. Inference of Reactive Rules from Dependency Models. RuleML. 2003.
Asaf Adi, Ayelet Biger, David Botzer, Opher Etzion and Ziva Sommer. Context Awareness in Amit. Active Middleware Services. 2003.
Asaf Adi, David Botzer, Opher Etzion and Tali Yatzkar-Haham. Monitoring Business Processes through Event Correlation based on Dependency Model. SIGMOD Conference. 2001.
Asaf Adi, David Botzer, Opher Etzion and Tali Yatzkar-Haham. Push Technology Personalization through Event Correlation. VLDB. 2000.
Opher Etzion. Complex Event Processing. ICWS. 2004.
Joris Mihaeli and Opher Etzion. Event Database Processing. ADBIS 2004 - Eighth East-European Conference on Advances in Databases. 2004.