Beware the market spam

In light of the news that Goldman Sachs’ Global Alpha — le quant fund extraordinaire –  has started “liquidating” its holdings, and that other algorithmically minded unwinds have probably been stalking the markets, we bring you the following chart from Nanex’s Eric Scott Hunsader:

What it shows is the incremental build-up over the last few years of superfluous quote traffic over the course of the day. That is to say, quotes which are entered into markets systems but are never actually transacted. Spam.
The more red, the more recent the data.
As Hunsader notes:

The chart shows how many quotes it takes to get $10,000 worth of stock traded in the U.S. for any point in time during the trading day over the last 4.5 years. Higher numbers indicate a less efficient market: it takes more information to transact the same dollar volume of trading. Quote traffic, like spam, is virtually free for the sender, but not free to the recipient. The cost of storing, transmitting and analyzing data, increases much faster than the rate of growth: that is, doubling the amount of data will result in much more than a doubling of the cost. For example, a gigabit network card costs $100, while a 10 gigabit network card costs over $2,000. A gigabit switch, $200; a 10 gigabit switch, $10,000. This October, anyone processing CQS (stock quotes) will have to upgrade all their equipment from gigabit to 10 gigabit. Which would be fine if this was due to an increase in meaningful data.
We think that a 10-fold increase in costs without any benefits would be considered “detrimental” by most business people.
This explosion of quote traffic relative to its economic value is accelerating. Data for September 14, 2011 is the thicker red line that snakes near the high. There is simply no justification for the type of quote data that underlies this growth. Only the computers spamming these bogus quotes benefit when they successfully trick other computers or humans into revealing information, or executing trades. This is not progress. Progress is almost always accompanied by an increase in efficiencies. This is completely backwards.

This needless to say produces a huge information burden, not only on anyone trying to trade the market, but also any regulatory systems trying to monitor and analyse what’s going on.
Back of an envelope guesstimates from our resident exchange sleuths regarding the data produced by US markets alone run into several hundred terabytes a year. That said, the higher the volatility the higher the data burden. In peak August volatility US markets supposedly generated as much as 1 terabyte of data per day, while previous peaks had been around 600 gigabytes. Add data from Europe and Asia and we could be hitting a figure of several petabytes a year.
It’s worth noting scientists at CERN in Switzerland faced a similar problem when building the Large Haron Collider. Though in their case they were faced with having to analyse roughly 15 petabytes (15m gigabytes) worth of data annually.
That’s the equivalent 1.7m dual-layer DVDs a year, they say.
Being scientists, they opted to get creative. A brand new computer network called “the grid” was the result, designed specifically to cope with information overload. Though to this day, even Cern doesn’t have the capacity to store all the data it produces and regularly chucks as much as it receives back into the abyss forever.
From a regulatory point of view, of course, chucking data away is not really the ideal solution for market data.
Meanwhile, it’s worth wondering how close we are to peak capacity right now.