John Bates of Progress explains how complex event processing works and how it can simplify the use of algorithms for finding and capturing trading opportunities.
A brief summary of Complex Event Processing
Complex Event Processing (CEP) is about treating actions that happen all the time as specific events, which describe the action, and then being able to analyze those events as they are streaming through a system, while looking through them for patterns that create opportunities or threats. In the trading world, this means things like trading opportunities, such as monitoring a set of instruments across multiple trading venues and looking for particular patterns. Those patterns might be high frequency trading (HFT), statistical arbitrage, correlation relationship between two items, or even execution algorithms that are slicing orders based on some predefined metric.
The threats often focus around pre-trade risk. For example, will placing the trade exceed predefined risk levels, or run into potentially abusive trades, like a wash trade. CEP is about being able to monitor business in real-time to analyze what is happening now and, based on that, to try to predict what is about to happen and act on it immediately.
The value of Complex Event Processing
The world of trading is so fast moving. Research done by the AITE Group suggests that the average lifespan of a trading algorithm can be as short as three months. This is because new trading patterns are constantly coming to light and ones that might have been very successful might no longer be available as the markets become more efficient. In the old days, trading algorithms were like a cottage industry, in much the same way as the making of muskets used to be. Highly paid and highly skilled craftsmen would handcraft the algorithm. It was the domain of the very rich and not very many could be involved in the game.
With the advent of CEP technologies in the last ten years, now anyone can find patterns in fast-flowing data feeds, but more importantly, CEP provides the tools for business people to describe new algorithms quickly. This means that traders can keep up with a trading world that is moving ever faster, and which the handmade craftsmen struggle to keep up with. Suddenly, it has become easier for smaller firms to create algorithms to compete with the larger ones. There has been a revolution in software for the trading space, in that firms of all sizes now have access to the technology that was previously available only to Tier 1 banks.
Peeking under the bonnet
In a CEP platform, there is an engine which has the tools that allow you to model and visualize new strategies as they are running, as well as see any opportunities or threats. On top of this is an adaptive layer, with connectors to convey different formats of events in and out of the processing engine, taking in market data and sending out trades. CEP platforms can work off a simple consolidated feed, but organizations find that it is better to connect to trading venues directly because it reduces the latency and things can be seen as they happen.
