While I was going through the list of enhancement requests for Camel-Extra, which is a community project related to Apache Camel, I came across an old request asking to support the default Esper configuration in order to ease the development of event patterns and queries. Camel-Extra is a sister project of ASF Camel that hosts components, which are not compliant to the Apache license. Within that space Esper is a LGPL licensed library that supports complex event processing (CEP) and analytics on event series. The the Camel component has been generously contributed to Camel-Extra by James Strachan in November 2007.
Although Esper possesses only a small number of configuration parameters, it is sometimes quite useful to simplify event patterns and event processing language (EPL) statements by providing a small amount of configuration parameters. Additionally, it might be useful to provide some tuning parameters meeting specific requirements. However, I am not tempted to offer a detailed description on how to configure the Esper engine in order to meet your specific requirements, since you will find a comprehensive guide in the Esper documentation. I will rather write on how to use the current camel-esper component.
Using Camel-Esper to Query Event Streams
Before diving into the configuration example, I would like to provide an overview, on how the camel-esper component can be configured in your route configuration in order to execute queries upon event streams without neglecting the fact, that you will always find the most recent documentation within the component description of Apache Camel. The component adheres to the overall Camel concept that defines a processing chain via an integration DSL calling subsystems via endpoint URI configurations. Esper can be considered within this context as one of those subsystems, which leads us to the fact that you need to configure an endpoint for being able to interact with the Esper library and framework.
Conceptual Overview: Calling Esper from Camel
Since Esper is embedded within Camel as component concept, addressing the component works like addressing any other subsystem of Camel. Fig. 1 provides an overview a route configuration which is being used as example. Route 1 has a direct endpoint, which serves as interface where any event producer can send messages in a synchronous invocation style. All messages being consumed from this endpoint will be passed towards an configured Esper endpoint, identified via a name as representing the internal ID within the Camel context that serves as addressable endpoint. The second part of the Esper configuration is a query or pattern piece, which does query the event streams coming through the specific Esper communication channel. Finally, after having executed the evaluation, the message will be passed within route 2 towards a consuming, direct endpoint.
Fig.1: Conceptual route configuration
This simple example shows already, that the logic to evaluate the event streams is encoded within the Esper endpoint. Esper offers basically two different options to write evaluation statements for event streams, a pattern language and an event query language. Both options have been integrated to the Esper component provided by camel-extra and will be introduced in the next two sections.
Esper Event Query Language Configuration
The first configuration example demonstrates, how it is possible to configure the camel-extra Esper component endpoint to query an event stream via the event processing language. The query language is a SQL like language, specifically designed to query event streams in contrast to database tables. The concept of stream therefore replaces the commonly known concept of tables. Nevertheless, since events are nothing else than data, the existing SQL concepts of joins, filtering and aggregation via grouping can be effectively applied upon streams as well.
|
1 2 3 4 5 6 7 8 9 |
public class EPLStockTickRouteBuilder extends RouteBuilder { public void configure() throws Exception { from("direct:start") .to("esper:stock"); from("esper:stock?eql=" + "select * from org.emmersberger.camelextra.esper.event.StockTick where symbol='AAPL'") .to("direct:apple"); } } |
In order to run your event queries based upon the event processing language, it is necessary to specify the eql option followed by the actual expression. In our case, we are looking for all events of type StockTick having the symbol AAPL which results in the select statement of the above endpoint configuration.
Esper Event Pattern Language Configuration
The second example shows a configuration option for using the event pattern language. The pattern language is based upon University research, originally conducted within the “Rapid” project at Stanford University. The Esper implementation is based upon dynamic state trees and can be considered as so called delta networks, where only changes to data is being communicated across object boundaries. Additionally, changes are only propagated, if the information is needed somewhere else. To optimise performance, Esper operates upon indices for data retrieval operations. The entire grammar of the pattern language is build on top of ANTLR, based on the Extended Backus-Naur Form (EBNF).
|
1 2 3 4 5 6 7 8 9 |
public class PatternStockTickRouteBuilder extends RouteBuilder { public void configure() throws Exception { from("direct:start") .to("esper:stock"); from("esper:stock?pattern=" + "every org.emmersberger.camelextra.esper.event.StockTick(symbol='AAPL')") .to("direct:apple"); } } |
To enable the Esper pattern language, it is required to define the pattern option in your endpoint configuration followed by the pattern expression you want to execute. In the example, we are looking for every Stock tick that contains the symbol ‘AAPL’, since we want to retrieve all information related to Apple.
One may have noticed, that addressing an event object requires the full package name, in order to acquaint Esper with the respective event type. Since typing the entire package name including the name of the actual Java object can be cumbersome, the following section introduces a different way, to induce the addressing of event types.
Enabling the File-based Configuration for Esper
Esper contains an option to provide configuration via an external XML-based file. The purpose of the configuration file is on the one hand to simplify queries, written in EPL and pattern language and on the other hand, to tune the engine behaviour to meet your individual requirements. Camel-extra’s Esper component supports the configuration via the default configuration file.
To enable the configuration via XML in camel-esper, it is required to set the configured option to true, a flag set per default to false. The specification of this parameter ensures, that camel-esper conducts a lookup for the esper.cfg.xml file in the root of your class path (e.g. <project_home>/src/main/resources/esper.cfg.xml).
|
1 2 3 4 5 6 7 |
public void configure() throws Exception { from("direct:start") .to("esper:stock"); from("esper:stock?pattern=" + "every StockTick(symbol='AAPL')&configured=true") .to("direct:apple"); } |
Having enabled the XML-based configuration, it is now possible to add a name for an event type and it’s corresponding class with fully qualified name (i.e. including the package name). This way, Esper knows, that the name refers to a specific address so that the name can be used within the EPL or pattern language. This example shows only a very limited set of configuration options provided by Esper. For a full reference, please refer to the configuration section of the Esper documentation.
|
1 2 3 4 5 6 7 |
<?xml version="1.0" encoding="UTF-8"?> <esper-configuration xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns="http://www.espertech.com/schema/esper" xsi:schemaLocation="http://www.espertech.com/schema/esper http://www.espertech.com/schema/esper/esper-configuration-4-0.xsd"> <event-type name="StockTick" class="org.emmersberger.camelextra.esper.event.StockTick"/> </esper-configuration> |
Summary
This article briefly introduces camel-esper, a component hosted within the camel-extra project. It outlines the integration and use of Esper within a camel route configuration at a conceptual level and demonstrates, how to adopt the EPL and pattern language, in order to select events from an event stream. Finally, one of the recently added features, of using the default Esper configuration via an XML file concludes the article and gives some insights in how to optimise your Camel application, when using Esper. Since the article introduces only the general usage concept, some further reading can be recommended.