Webinar: Event Processing for Java Developers

I'll be speaking at a webinar Event Processing for Java Developers arranged by Starview Technology, Inc tomorrow (Nov 9) at 10am PST. Register here.

Introducing an Event Server Platform

After working for about seven years shaping and evangelizing for a Java Application Server platform and J2EE, I decided to move on. For the last year or so I’ve been spending my energy shaping up the next-generation middleware platform for building, deploying, and managing event-processing applications.

Most vendors have focused on providing limited functionality such as windowing, filtering, and pattern matching, often known as Complex Event Processing (CEP).These vendors have also been targeting limited use cases in the Financial Services sector. As CEP did not take off, many vendors have buried their offerings inside their SOA and BPM solutions. However, I think event processing is pervasive inside all businesses. Whenever you tweet or send a text/sms it generates an event. Whenever you plug in your electric vehicle to the grid, whenever you have a power outage, or whenever a machine breaks down in a factory – events are generated. A smart business needs to analyze and exploit these messages to make the right decision to take the right decision at the right time. Many of the new generation of applications are being built using an event-driven paradigm and need a new generation of middleware platform named an Event Server Platform. In this article, I will introduce an event server platform.

What is an Event Server?

Why do you use an application server? Because you do not want to reinvent the wheel and take advantage of several services the application server provides to quickly build your application. An event server provides similar functionality for users to rapidly build and deploy event-processing applications – optimized for event processing. I will discuss why traditional application servers are not suitable for event processing in a future article. One of the key points here is that traditional application servers are optimized for request-response applications and not for event processing.

In all practical senses an event server is an application server optimized for event processing applications. Let us look at an example architecture. The following figure shows the architecture for the Starview Event Server that is built on OSGi:

You have to build an application before you deploy it to an event server. So you need tools and languages to build an application.

Development Tool

You will need to build, test, and debug your event-driven application and hence you will need an IDE. Here is an example of Starview ACE that uses a model-driven approach to build an event-driven application. Starview ACE is an Eclipse plug-in and application models are based on the Eclipse Model Framework:

Connectivity Adapters

You will need to capture an event stream at its source and in-bound adapters provide this connectivity. The event source can be a messaging system, SNMP traps, socket reader, log files, database updates, and so on. An event server provides out–of-the-box adapters to simplify reading event sources without much programming. The adapters also generate outbound events or integrate with third-party systems and resources for correlating events.

Programming Language aka Event Processing Language

In order to process the events you need an Event Processing Language. The CEP vendors often refer to Stream SQL as their EPL. However, as you know SQL is quite limiting in nature and you will need the full semantics of a programming language built for event processing that provides fast and efficient in-memory structures to represent complex data types, andin-stream processing and analytics. The Event Processing Language must provide the ability to maintain state and support the concept of an event-processing agent for implementing complex event-processing rules.

Also, you do not want your event-processing rules to be static in nature: you want to enable your business users to author rules. Hence the programming language must provide a foundation to develop Domain-Specific Languages.

Here is a typical architecture for such an Event Processing Language:

This diagram shows the architecture for the Star™ language.

You may ask, “where is Java in this equation?” The event servers must integrate with existing Java applications, and provide the ability to build applications using Java. You have to remember, though, that Java has its limits and you have to explore the capabilities provided by Event Processing Languages.

Distributed Application

Many of the event processing applications are distributed in nature and require event processing at the edge. These are prevalent in many use cases such as Quality of Service, Smart Grid optimization, and manufacturing automation, where you want to process events locally and filter out unnecessary events at the edge. The event server platform must provide mechanisms to deploy a lightweight version of the event server at the edge and collaborate with a centralized event server without requiring hundreds of lines of code!

Management Infrastructure

You need a good management infrastructure for managing your event servers and applications. This becomes challenging when applications are distributed in nature. The management infrastructure should provide the ability to deploy, manage, and monitor applications, event servers, and server groups. And the infrastructure must be built using an event-driven paradigm.

The following screen shot shows the management console for Starview Enterprise Hub that provides such a management infrastructure:

These are the basics of an Event Server Platform. You will several advanced features such as high-availability, caching, etc.

We will discuss some of these topics in detail in future blog entries.

Bye!
References and Suggested Reading

• Event Processing In Action
• Starview Enterprise Platform
• Attend my web seminar on Nov 9, 2010 Event Processing for Java Developers