By Hellinton H. Takada, Ph.D. Vice President of Quantitative Research at Itaú Asset Management and Tiago M. Magalhães, Ph.D. Senior Analyst of Quantitative Research at Itaú Asset Management
Undoubtedly, the volume weighted average price (VWAP) is widely used as an industry standard approach to measure equity execution performance. The average executed price is usually compared with the corresponding VWAP benchmark. In this context, our objective is the introduction and application of a statistical procedure to measure more adequately the performance of equity trades using VWAP as a benchmark. However, it is also important to have in mind that depending on the execution procedure and the performance evaluation purposes, it is necessary to have other benchmarks than VWAP. For instance, when the opportunity cost is very relevant or the execution strategy is opportunistic, it becomes important to consider the implementation shortfall.
The measurement of the execution performance is imperative due to regulators’ and investors’ requirements related to transaction cost analysis (TCA) and best execution. However, there are many difficulties related to empirical researches on the subject. Obviously, the lack of public data showing the trades per execution makes the studies scarcer. Additionally, the underlying execution strategy must be comparable with VWAP. Consequently, we focus on our private database of Brazilian equities’ trades using third-party VWAP execution algos. The idea is to show the aggregate performance of such algos from several brokerage firms to give an overview of the local VWAP execution services and, at the same time, to illustrate our statistical methodology based on a statistical bootstrapping methodology.
In our dataset of executed trades using VWAP algos, there are more than 10,000 observations from 2014 until the first quarter of 2016. As expected, there is a high autocorrelation structure between observations belonging to the same day, that is, the executed trades over a day are slices of a very larger order. Since correlated data make the statistical analyses more complicated, as our first step we break such autocorrelations aggregating per day the financial values of all the individual trades. Using statistical tests, it is possible to verify the independence of the obtained daily financial values. It is highly probable that there are many larger orders being executed over several days. However, at least empirically, the autocorrelation caused by them seems to be negligible when they are mixed with other execution flows.
Basically, our performance metric is the financial result in basis points over the VWAP benchmark. Usually the transaction costs of the investment strategies are all calculated on an annual basis. Consequently, it makes sense to also present the performance of the executions on an annual basis. Another point to consider is that the distribution of the daily metric possesses a higher uncertainty than the distribution of the annual metric. In Figure 1, we show the obtained cumulated distributions of the daily and annual metrics. The details related to the statistical procedure to obtain the graphs are presented in the next paragraphs. Yet, it is common to see just the publication of the mean or the median of the daily metrics leading to erroneous inferences.
Actually, the publication of the entire distribution of the metric is of utmost importance.
It is important to remember that we have the daily metric from the aggregation of the individual trades to avoid autocorrelation and, consequently, we also have the distribution of the daily metric. Unfortunately, the calculation of the distribution of the annual metric using the distribution of the daily metric is not so straightforward because the annual metric is a random variable equal to a ratio of sums of daily random variables. Furthermore, the annual aggregation of the individual trades is useless because we do not have several years to obtain the annual distribution. The alternative is the use of a nonparametric method where we do not impose any specific structure for the related problem. Consequently, our objective is to avoid making any additional assumptions to obtain the desired annual distribution.
Bootstrapping
Particularly, we adopt the bootstrapping method from statistics. In statistics, bootstrapping refers to any test or metric that relies on random sampling with replacement from a predefined dataset. This technique allows estimation of the sampling distribution of almost any statistic using random sampling methods, that is, a bootstrapping method lets us obtain useful information such as the statistical moments (mean, standard deviation, etc.) and percentiles (median, quartiles, etc.) of the distribution of interest. Obviously, the number of samplings must be enough to obtain a distribution that does not change significantly after adding new samplings. The cumulated distributions obtained using the bootstrapping method for the daily and annual metrics are presented in Figure 1. As previously mentioned, the distribution of the daily metric has a much higher uncertainty than the distribution of the annual metric.
As a final remark, an interesting empirical observation using our private dataset is that the median and mean of the daily metric distribution are slightly negative. On the other hand, the same statistics of the annual metric distribution are positive. Consequently, the VWAP execution algos from our dataset generate alpha in relation to the VWAP benchmark on an annual basis. Unfortunately, it is not possible to observe such alpha in a daily basis. It is interesting that the accumulation of the daily metrics with negative median and mean results in an annual metric distribution with positive median and mean. Finally, it is also clear that the annual metric distribution possesses a positive asymmetry meaning that there is a higher probability of the occurrence of extreme positive outcomes than extreme negative outcomes.
We’d love to hear your feedback on this article. Please click here
