Release of Camel-Extra 2.14.0 and LGPL License Support

Only few people may have recognized that June 21st 2015 was a big day for the Camel-Extra community. Besides the fact that this day in June marks the first release of Camel-Extra that supports ASF Camel 2.14.x, it is the release where we opened up the license support to the most open OSS license that is applicable for the underlying component.

Free and Open Source Software

Free and Open Source Software

Finding the appropriate software license is always difficult for an open source project. This statement is even more valid if you start building off existing components where each has its own license model. Since Camel-Extra extends the enterprise integration project ASF Camel with components that can not be hosted within the Apache infrastructure, it is built into its DNA to deal with third-party libraries. The reason why Camel-Extra components can not be hosted within the Apache infrastructure is related to license compatibility issues between the Apache License and the third-party OSS licenses.

Courageous Camel riders have therefore built an environment within Apache-Extras, to provide an environment where the development and community support for these components is possible. In order to simplify the build structure and reduce the amount of headache with new components, the initial attempt was to base Camel-Extra upon the GNU General Public License. However, this approach lead to several discussions within the community, since the GPL License is not very friendly for commercial adoption. As a consequence, we’ve decided to approach a multi-license support strategy, which allows us to open the software license for components that are not dependent upon GPL libraries. On the other hand, those components that have still dependencies to GPL libraries still reside under its respective parent license.

The following list will help to understand the current license assignments. Please note that these assignments may change over time, in case the underlying library adjust their license model.

GPL License LGPL License
camel-db4o camel-couchbase
camel-esper camel-exist
camel-spring-neo4j camel-hibernate
camel-vtdxml camel-jboss
camel-jboss6
camel-jcifs
camel-rcode
camel-virtualbox
camel-zeromq

I hope you all enjoy the new license approach and many of you will be able to adopt components within your projects. Please remember that we love contributions. Anything you share with the community (i.e.filing bugs, contributing code, helping with documentation) will help to maintain the project better!

Esper Component Configuration and Config File Support in Camel-Extra

While I was going through the list of enhancement requests for Camel-Extra, which is a community project related to Apache Camel, I came across an old request asking to support the default Esper configuration in order to ease the development of event patterns and queries. Camel-Extra is a sister project of ASF Camel that hosts components, which are not compliant to the Apache license. Within that space Esper is a LGPL licensed library that supports complex event processing (CEP) and analytics on event series. The the Camel component has been generously contributed to Camel-Extra by James Strachan in November 2007.

Although Esper possesses only a small number of configuration parameters, it is sometimes quite useful to simplify event patterns and event processing language (EPL) statements by providing a small amount of configuration parameters. Additionally, it might be useful to provide some tuning parameters meeting specific requirements. However, I am not tempted to offer a detailed description on how to configure the Esper engine in order to meet your specific requirements, since you will find a comprehensive guide in the Esper documentation. I will rather write on how to use the current camel-esper component.

Using Camel-Esper to Query Event Streams

Before diving into the configuration example, I would like to provide an overview, on how the camel-esper component can be configured in your route configuration in order to execute queries upon event streams without neglecting the fact, that you will always find the most recent documentation within the component description of Apache Camel. The component adheres to the overall Camel concept that defines a processing chain via an integration DSL calling subsystems via endpoint URI configurations. Esper can be considered within this context as one of those subsystems, which leads us to the fact that you need to configure an endpoint for being able to interact with the Esper library and framework.

Conceptual Overview: Calling Esper from Camel

Since Esper is embedded within Camel as component concept, addressing the component works like addressing any other subsystem of Camel. Fig. 1 provides an overview a route configuration which is being used as example. Route 1 has a direct endpoint, which serves as interface where any event producer can send messages in a synchronous invocation style. All messages being consumed from this endpoint will be passed towards an configured Esper endpoint, identified via a name as representing the internal ID within the Camel context that serves as addressable endpoint. The second part of the Esper configuration is a query or pattern piece, which does query the event streams coming through the specific Esper communication channel. Finally, after having executed the evaluation, the message will be passed within route 2 towards a consuming, direct endpoint.

Fig.1: Conceptual route configuration

Fig.1: Conceptual route configuration

This simple example shows already, that the logic to evaluate the event streams is encoded within the Esper endpoint. Esper offers basically two different options to write evaluation statements for event streams, a pattern language and an event query language. Both options have been integrated to the Esper component provided by camel-extra and will be introduced in the next two sections.

Esper Event Query Language Configuration

The first configuration example demonstrates, how it is possible to configure the camel-extra Esper component endpoint to query an event stream via the event processing language. The query language is a SQL like language, specifically designed to query event streams in contrast to database tables. The concept of stream therefore replaces the commonly known concept of tables. Nevertheless, since events are nothing else than data, the existing SQL concepts of joins, filtering and aggregation via grouping can be effectively applied upon streams as well.

In order to run your event queries based upon the event processing language, it is necessary to specify the eql option followed by the actual expression. In our case, we are looking for all events of type StockTick having the symbol AAPL which results in the select statement of the above endpoint configuration.

Esper Event Pattern Language Configuration

The second example shows a configuration option for using the event pattern language. The pattern language is based upon University research, originally conducted within the “Rapid” project at Stanford University. The Esper implementation is based upon dynamic state trees and can be considered as so called delta networks, where only changes to data is being communicated across object boundaries. Additionally, changes are only propagated, if the information is needed somewhere else. To optimise performance, Esper operates upon indices for data retrieval operations. The entire grammar of the pattern language is build on top of ANTLR, based on the Extended Backus-Naur Form (EBNF).

To enable the Esper pattern language, it is required to define the pattern option in your endpoint configuration followed by the pattern expression you want to execute. In the example, we are looking for every Stock tick that contains the symbol ‘AAPL’, since we want to retrieve all information related to Apple.

One may have noticed, that addressing an event object requires the full package name, in order to acquaint Esper with the respective event type. Since typing the entire package name including the name of the actual Java object can be cumbersome, the following section introduces a different way, to induce the addressing of event types.

Enabling the File-based Configuration for Esper

Esper contains an option to provide configuration via an external XML-based file. The purpose of the configuration file is on the one hand to simplify queries, written in EPL and pattern language and on the other hand, to tune the engine behaviour to meet your individual requirements. Camel-extra’s Esper component supports the configuration via the default configuration file.

To enable the configuration via XML in camel-esper, it is required to set the configured option to true, a flag set per default to false. The specification of this parameter ensures, that camel-esper conducts a lookup for the esper.cfg.xml file in the root of your class path (e.g. <project_home>/src/main/resources/esper.cfg.xml).

Having enabled the XML-based configuration, it is now possible to add a name for an event type and it’s corresponding class with fully qualified name (i.e. including the package name). This way, Esper knows, that the name refers to a specific address so that the name can be used within the EPL or pattern language. This example shows only a very limited set of configuration options provided by Esper. For a full reference, please refer to the configuration section of the Esper documentation.

Summary

This article briefly introduces camel-esper, a component hosted within the camel-extra project. It outlines the integration and use of Esper within a camel route configuration at a conceptual level and demonstrates, how to adopt the EPL and pattern language, in order to select events from an event stream. Finally, one of the recently added features, of using the default Esper configuration via an XML file concludes the article and gives some insights in how to optimise your Camel application, when using Esper. Since the article introduces only the general usage concept, some further reading can be recommended.

  1. Apache Camel Component Concept
  2. Camel-Extra Esper Component
  3. Camel-Extra Project
  4. Esper Event Processing Language
  5. Esper Pattern Language
  6. Esper Configuration