Simple Binary Becomes A FIX Standard

Share

By Don Mendelson, Owner, Silver Flash LLC and Technical Architect, FIX Trading Community

SBE has led to both a technical and an organizational revolution for FIX and is increasingly being adopted throughout the financial industry.

For over two decades, FIX Protocol has been encoded on the wire in more-or-less humanly readable formats. The original FIX format, called tag=value encoding, is still the world-wide standard in trading messages. It was followed several years later by FIXML, which predominates in clearing. That was the state-of-the-art until it crashed into a contradiction of high performance trading.

Very simply, computers cannot operate directly on humanly readable character data. Rather, they calculate using binary numbers and data structures. Something had to change to avoid latency introduced by the unnecessary translation between character and binary data.

Several venues created proprietary message formats to solve the problem. But the FIX Trading Community is in the standards business. It would be to any trading firm’s advantage to have a FIX standard for binary messages rather than having to invest in supporting numerous proprietary formats. Therefore, the FIX organization launched a project to create such a standard. It was ultimately named Simple Binary Encoding (SBE).

SBE is a FIX standard—any existing FIX message can be encoded with it and gain some benefit of low latency processing. At the same time, it conveys the same business semantics that people are familiar with. The huge knowledge base that firms have of FIX messages and fields still applies. Software developers need to learn the details of the encoding, but business leaders can still rely on what they already know about applications.

Breaking down barriers to participation
The SBE effort led not only to a technical revolution for FIX, but also an organizational one. Although the FIX Trading Community has broad membership in the financial industry, it was seen by some technologists as a closed system. Many small trading firms and vendors do not have the resources to participate in a standards organization, but they can benefit from transparency and a way of making their voices heard. Therefore, the FIX Global Technical Committee (GTC) made a conscious decision to break down barriers to participation in technical standards development.

One decision was to make FIX technical standards projects open to the public in GitHub, the one-stop shop for open-source code. Projects are created there for new standards and for demonstration code to get developers started. Any software developer can obtain their own user ID from GitHub for free. They do not need to be members of FIX Trading Community, nor does the FIX organization need to grant them rights to see the projects.

That is as transparent as you can get. Not only that, but anyone can enter issues in the GitHub issue tracker to make corrections or request enhancements. They can even propose changes to standards or contribute new features in code, through what is called a pull request in GitHub.

Another aspect of opening standards is publishing them with clear legal rights to copyrighted material. FIX technical standards are published with Creative Commons Attribution-NoDerivatives 4.0 International License.

To summarize, the license says that anyone can republish the material so long as it is attributed to FIX Trading Community and the standard is not altered. (An altered standard is not a standard.) FIX demonstration code is open-sourced under the liberal Apache License, Version 2.0. It grants rights to developers to use and alter the code as they see fit, so long as they attribute it to the original author.

Development process
Each new FIX technical standard is developed by a working group that is formed by the GTC. Some of the groups, such as the High Performance Working Group, have a number of standards under their wing, so they are divided into subgroups. There is a subgroup for Simple Binary Encoding, as well as several other message encodings and session-layer standards under development. Each working group is supplemented by anyone in the world who wants to participate through GitHub.

Another important decision was establishing a documented process for developing and approving FIX technical standards. The process follows the proven industry trend of agile project development. Its most important principle is iterative development.

It is impossible to foresee every impact or realize every requirement at the beginning of a project. Rather, new information and requirements are discovered by doing. In the FIX process, a milestone is reached by publishing a release candidate, an interim draft of a standard. Each release candidate either adds features or refines the work of earlier iterations. In the case of SBE, it went through four such iterations before reaching final form.

Each release candidate is published on a FIX web site as well as in GitHub for a period of public review.

Typically, the public review period for a release candidate is 90 days. User comments may be entered either in a forum on a FIX site or in GitHub. They are taken as input to be considered for a possible next iteration.

Criteria to become a standard
When a working group is satisfied with its product, then it recommends promotion to draft standard. There are two requirements that a draft standard must meet to be accepted.

First, the public review period for a draft standard is extended to at least six months to make sure that all voices have been heard. Second, there must be at least two interoperable implementations of the proposed standard.

When these requirements are met, the working group recommends promotion to final technical standard. The GTC must approve. After that, the standard is immutable. However, if there is interest, subsequent versions of the standard may be initiated.

In the case of SBE version 1.0, it reached the goal of technical standard in February 2017. It met the requirements for an extended public review, but also multiple implementations were created by different authors. To prove interoperability, the GTC sponsored development of a conformance test suite. Like the SBE specification, the conformance project is open-source and resides in GitHub. Any implementer can run the test on their own SBE project while it is under development. And then they can announce conformance when work is completed if they want to share it with the world.

SBE has been implemented in C++, Java, Golang and Python programming languages. Due to its success, other implementations are likely to follow. It has proven its worth. A group that created the best-known SBE implementation claims that it can encode or decode messages about 20 times faster than a well-known message encoding put out by one of the information industry giants.

One of the early adopters was CME Group. It uses SBE for market data and internal messaging. Moscow Exchange is using SBE for derivatives messaging. Thomson Reuters has made SBE one of its supported

API formats, and a growing number of financial industry users employ it. It has even gained interest outside the financial industry due to its performance advantages.

SBE version 1.0 was the first FIX technical specification to reach final approval under the process. Work has already commenced to explore a possibly enhanced version 2.0.

We’d love to hear your feedback on this article. Please click here