24.10.07
Humans versus black boxes
Beware of quants? Beware of geeks bearing greeks? Some models don't work therefore ALL models don't work? It is curious how fear and hype usually lead to unjustified generalizations. Semantic pigeon holing is a common response to anything unfamiliar. Presumably that's why so many think quants are in a quandry, derivatives are dangerous and leverage is lunacy. Black boxes are secret and ALL the same(!) so better to stay with "human" methods of making money? There are many more bad human investors out there than bad quant systems so beware of the QUALS as much as the QUANTS.
Investing successfully is hard. It makes sense to use all available tools. A systematic, replicable investment process using qualitative AND quantitative analysis is surely the bedrock of any successful hedge fund, though how they weight the two disciplines may vary. The simple fact is there are GOOD pricing and trading models around and there are BAD ones. It usually takes bear markets and volatility to show which is which. But whether the models produce positive or negative alpha is entirely up to human inputs. Garbage in, garbage out or quality in, quality out.
I think investors should be beware of everything. Considering the non-quant problems and dire risk management policies on display recently, this faith in the value of human discretion seems ironic. Sure there are plenty of poorly designed and badly tested quant trading systems out there as there are delusional pricing models but that does not preclude the existence of quality, robust products. A computer making the trading decisions rather than a human does NOT mean an increase in systemic risk or a decrease in the persistence of a good strategy. It just puts the emphasis on ensuring the computer is making decisions in a different way to other computers.
The main distinction comes down to whether the human decides or the computer, programmed by humans, decides. But is that really a distinction? If a systematic trading model needs adjusting to "new" phenomena then it wasn't properly tested in the first place. Why the fuss about "ALL" quant funds? It probably comes down to the fear of the unknown and hatred of opacity. Discretionary investors can be reasonably open about how they pick stocks since the "edge" comes down to the skill in implementing the strategy. Good systematic strategy inventors cannot be so open since 1) the edge is the strategy 2) no-one outside quant land will understand 3) those inside quant land will steal it leading to the inefficiency disappearing and trade crowding problems.
Any successful investment strategy needs a robust decision-making framework and elimination of emotions. The best way is a division of labor between humans that are good at gathering data and machines that are good at processing that data in the way a human asks them to do. The more short term the trading the more useful artificial intelligence is going to be. It makes sense that PROVIDED the algorithm has been put together competently to ask the computer to trade IF time is of the essence. High frequency trading is very dependent on low latency and incorporating a human override slows thing down enormously. With high frequency the speed of execution and reduction of slippage often IS the edge.
There is nothing new about bad quantitative models running into problems. There have been a few post mortem of the so-called quant meltdown as if it was the first time this had occurred. But it is similar to the portfolio insurance of 1987, the mortgage-backed securities pricing "models" of 1994 or LTCM's quant and option pricing "geniuses" in 1998. Just as there are good and bad human stock-pickers, there are good and bad human quants. Show me a human based strategy that hasn't also run into problems at some time.
Just because public domain quant strategies using the same methodologies will identify the same stories and opportunities does not invalidate other proprietary methods. The quant methodologies that ran into trouble - 1) find pairs of stocks historically cointegrated and take the other side when they are X sigma apart or 2)throw every fundamental and technical variable you can think of into the hopper and data mine for what patterns worked in the past - are now very crowded. Apart from some now very large hedge funds (a few good, many bad) there were investment bank proprietary desks heavily in the statistical arbitrage and factor model strategies.
Some multistrategy hedge funds that couldn't unwind illiquid credit instruments were forced to unwind what was liquid to meet margin calls. The situation was exacerbated by the 130/30 asset gathering crowd panicking when their shorts began to tick up on all the short covering. I wonder how many of them knew beforehand that short positions get bigger as you lose money. I wouldn't be surprised if some of the less experienced 130/30 entrants were temporarily more like 120/40 or even 110/50 in early August. Even "unleveraged" funds are leveraged when you short sell.
Models are only as good as the assumptions humans give them and the programmer's representation of reality. Unfortunately reality is rather complicated. To put it mildly the facts have not been kind to the theories. If you code up some C++ or C# and tell the computer that we live in a nice "normal", "standard" world of rational entities that spend their days maximizing their utility and immediately changing prices accurately to new information then you will run into trouble. The computer only knows things that YOU choose to let the computer know about. If you lose money beyond statistical expectation then that is a human error, not computer error. Try typing =850*77.1 into Microsoft Excel 2007. Is it a human design error or the computer's fault if you get 100,000 instead of the correct 65,535?
Computers are just a tool. Humans design "discretionary" investment strategies and they design "systematic" investment strategies. If they are good or bad is all up to human ingenuity or human stupidity. Whether they data mine the past or test hypotheses of the future is up to human skill. Computers are good at information processing but can only analyse the data they are given in the way humans designate. Quantitative risk management is only possible based on the factors input to the system; if the machine has a blindspot to a new factor there will be errors often of a non-linear order of magnitude. Computers are simple creatures; if you only tell them about bell-curves and the "rarity" of 6 sigma moves then they are not going to perform very well when 25 sigma moves come along.
There are no axioms or proofs in real world markets. Asset classes don't read the textbooks unfortunately. An IQ test can be coached but the market is an IQ test where the questions and answers change while you are taking it. If you assume randomness and rationality on a deterministic chaotic process like the markets then your models are going to be wrong. As we have seen recently everything is connected so therefore models that rely on independence are headed for trouble. A good model is one that provides a persistent trading or pricing edge, can cope with a non-linear, dependent, varying factor world and whose underlying theory and equations have NEVER been published. Some quants like to use something called "stochastic calculus" which is useful for many things but certainly NOT financial modelling.
As simple tools computers are not good at complex event analysis because most programming hasn't focused on that area. Unless its human owner has informed it that most CDO pricing is wrong, that if Andy defaults then the chance of Bob and Chuck also defaulting is much higher than the credit models "assume", and that there are a bunch of other people out there running very similar equity mean reversion programs, then the model won't pick up that maybe it should change things. It just follows orders.
If quants neglect to inform their computers that if a weaker player with a similar model is forced to unwind then the opposite of what "should" happen might occur then that is also human error. If the computer doesn't know that liquidity is very variable and can even evaporate then whose fault is that omission? CDO and CLO mispricing was primarily based on the Gaussian copula model. Quick investment tip: never, ever risk money on anything with the word "Gaussian" in it. Gaussian things make the mathematics easy which is why they don't work. Bank CEOs might bear that in mind; there are quite a few careers still being bet on the multivariate normal curve.
However gatekeepers who avoid ALL quant strategies are doing a poor job for their clients. Intermediaries should earn their fees by identifying the good and bad in a strategy NOT avoiding it altogether. Process driven investment decisions are the foundation of EVERY robust strategy. Given the large amounts of data that silicon based computers can analyse that their carbon based masters decide to feed them, it makes sense to outsource such work to them. Even odder are those investors who make an allocation to "quant" and then refrain from any other quant funds. As if all quant strategies were the same!
One of the biggest risks for any hedge fund is that its human assets walk out of the door each day. Computers can absorb and react to information on a 100,000 securities instantly unlike a human trader. They can analyse new data, have the order in and executed before a human brain has even noticed the information. Computers don't think they are a genius when they fluke a lucky trade. They don't take lunch or vacations. They don't quit and try to set up their own fund with proprietary information. They don't have clandestine meetings with competitors. They don't complain about colleagues, clients, salaries and bonuses. They don't get sick or crash their Porsches. They don't lose interest after the IPO. And once the programming and testing is done you have eliminated key-person risk. There is a lot in favor of purely systematic strategies IF they are good.
Few investment managers admit to using that big institutional no-no called technical analysis despite the fact that many do. But calling it quantitative analysis is still ok, just. Computing power allows detection of predictive patterns and structure; we've long moved on from public domain moving averages, breakouts, candlesticks, RSI and MACD. Technical analysts look at patterns of prices and volume while fundamental analysts look at patterns of earnings and book value. Growth investors are trend followers while value investors are basically countertrend. Are fundamental analysis and technical analysis that far apart or is it just a change of inputs to the model?
Even if you buy into the "normal" nonsense, 95% Var means that about 1 day every month on "average" you will lose more. While $480 million losses may look bad, on $10 billion notional it is only 4.8%. If the Morgan Stanley quants made the human decision to run $2 billion notional cash at 5x leverage, losses of that magnitude, while bad, are not completely beyond the realms of expectation. The valuation noise on large portfolios is going to be tens or hundreds of millions even in relatively quiet times let alone market stress. Strangely no heads have rolled, yet, at Goldman Sachs' Global Negative Alpha "hedge fund" despite squandering over $3 billion of client money on its disastrous factor "models". $8.4 billion losses, mostly due to buying market share in CDOs and structured credit with little concept of risk, are another matter. The Merrill Lynch losses were due to human decisions.
Computers are at the mercy of what data their programmers choose to give them. Even genetic algorithms and neural nets rely on the system parameters and data sets provided by humans. Computers have solved simple finite systems like a chess game because it is a closed and rational problem. There is always an optimal move in any situation. But financial markets are much more complex, require decision making under uncertainty and the rules change while you are playing the game. We are a long time away from artificially matching the kinaesthetic intelligence of a basketball or soccer player. The intelligence necessary to master ball games is far beyond that required for board games. In financial markets computers are just a basic aid to human decision-making.
Investing successfully is hard. It makes sense to use all available tools. A systematic, replicable investment process using qualitative AND quantitative analysis is surely the bedrock of any successful hedge fund, though how they weight the two disciplines may vary. The simple fact is there are GOOD pricing and trading models around and there are BAD ones. It usually takes bear markets and volatility to show which is which. But whether the models produce positive or negative alpha is entirely up to human inputs. Garbage in, garbage out or quality in, quality out.
I think investors should be beware of everything. Considering the non-quant problems and dire risk management policies on display recently, this faith in the value of human discretion seems ironic. Sure there are plenty of poorly designed and badly tested quant trading systems out there as there are delusional pricing models but that does not preclude the existence of quality, robust products. A computer making the trading decisions rather than a human does NOT mean an increase in systemic risk or a decrease in the persistence of a good strategy. It just puts the emphasis on ensuring the computer is making decisions in a different way to other computers.
The main distinction comes down to whether the human decides or the computer, programmed by humans, decides. But is that really a distinction? If a systematic trading model needs adjusting to "new" phenomena then it wasn't properly tested in the first place. Why the fuss about "ALL" quant funds? It probably comes down to the fear of the unknown and hatred of opacity. Discretionary investors can be reasonably open about how they pick stocks since the "edge" comes down to the skill in implementing the strategy. Good systematic strategy inventors cannot be so open since 1) the edge is the strategy 2) no-one outside quant land will understand 3) those inside quant land will steal it leading to the inefficiency disappearing and trade crowding problems.
Any successful investment strategy needs a robust decision-making framework and elimination of emotions. The best way is a division of labor between humans that are good at gathering data and machines that are good at processing that data in the way a human asks them to do. The more short term the trading the more useful artificial intelligence is going to be. It makes sense that PROVIDED the algorithm has been put together competently to ask the computer to trade IF time is of the essence. High frequency trading is very dependent on low latency and incorporating a human override slows thing down enormously. With high frequency the speed of execution and reduction of slippage often IS the edge.
There is nothing new about bad quantitative models running into problems. There have been a few post mortem of the so-called quant meltdown as if it was the first time this had occurred. But it is similar to the portfolio insurance of 1987, the mortgage-backed securities pricing "models" of 1994 or LTCM's quant and option pricing "geniuses" in 1998. Just as there are good and bad human stock-pickers, there are good and bad human quants. Show me a human based strategy that hasn't also run into problems at some time.
Just because public domain quant strategies using the same methodologies will identify the same stories and opportunities does not invalidate other proprietary methods. The quant methodologies that ran into trouble - 1) find pairs of stocks historically cointegrated and take the other side when they are X sigma apart or 2)throw every fundamental and technical variable you can think of into the hopper and data mine for what patterns worked in the past - are now very crowded. Apart from some now very large hedge funds (a few good, many bad) there were investment bank proprietary desks heavily in the statistical arbitrage and factor model strategies.
Some multistrategy hedge funds that couldn't unwind illiquid credit instruments were forced to unwind what was liquid to meet margin calls. The situation was exacerbated by the 130/30 asset gathering crowd panicking when their shorts began to tick up on all the short covering. I wonder how many of them knew beforehand that short positions get bigger as you lose money. I wouldn't be surprised if some of the less experienced 130/30 entrants were temporarily more like 120/40 or even 110/50 in early August. Even "unleveraged" funds are leveraged when you short sell.
Models are only as good as the assumptions humans give them and the programmer's representation of reality. Unfortunately reality is rather complicated. To put it mildly the facts have not been kind to the theories. If you code up some C++ or C# and tell the computer that we live in a nice "normal", "standard" world of rational entities that spend their days maximizing their utility and immediately changing prices accurately to new information then you will run into trouble. The computer only knows things that YOU choose to let the computer know about. If you lose money beyond statistical expectation then that is a human error, not computer error. Try typing =850*77.1 into Microsoft Excel 2007. Is it a human design error or the computer's fault if you get 100,000 instead of the correct 65,535?
Computers are just a tool. Humans design "discretionary" investment strategies and they design "systematic" investment strategies. If they are good or bad is all up to human ingenuity or human stupidity. Whether they data mine the past or test hypotheses of the future is up to human skill. Computers are good at information processing but can only analyse the data they are given in the way humans designate. Quantitative risk management is only possible based on the factors input to the system; if the machine has a blindspot to a new factor there will be errors often of a non-linear order of magnitude. Computers are simple creatures; if you only tell them about bell-curves and the "rarity" of 6 sigma moves then they are not going to perform very well when 25 sigma moves come along.
There are no axioms or proofs in real world markets. Asset classes don't read the textbooks unfortunately. An IQ test can be coached but the market is an IQ test where the questions and answers change while you are taking it. If you assume randomness and rationality on a deterministic chaotic process like the markets then your models are going to be wrong. As we have seen recently everything is connected so therefore models that rely on independence are headed for trouble. A good model is one that provides a persistent trading or pricing edge, can cope with a non-linear, dependent, varying factor world and whose underlying theory and equations have NEVER been published. Some quants like to use something called "stochastic calculus" which is useful for many things but certainly NOT financial modelling.
As simple tools computers are not good at complex event analysis because most programming hasn't focused on that area. Unless its human owner has informed it that most CDO pricing is wrong, that if Andy defaults then the chance of Bob and Chuck also defaulting is much higher than the credit models "assume", and that there are a bunch of other people out there running very similar equity mean reversion programs, then the model won't pick up that maybe it should change things. It just follows orders.
If quants neglect to inform their computers that if a weaker player with a similar model is forced to unwind then the opposite of what "should" happen might occur then that is also human error. If the computer doesn't know that liquidity is very variable and can even evaporate then whose fault is that omission? CDO and CLO mispricing was primarily based on the Gaussian copula model. Quick investment tip: never, ever risk money on anything with the word "Gaussian" in it. Gaussian things make the mathematics easy which is why they don't work. Bank CEOs might bear that in mind; there are quite a few careers still being bet on the multivariate normal curve.
However gatekeepers who avoid ALL quant strategies are doing a poor job for their clients. Intermediaries should earn their fees by identifying the good and bad in a strategy NOT avoiding it altogether. Process driven investment decisions are the foundation of EVERY robust strategy. Given the large amounts of data that silicon based computers can analyse that their carbon based masters decide to feed them, it makes sense to outsource such work to them. Even odder are those investors who make an allocation to "quant" and then refrain from any other quant funds. As if all quant strategies were the same!
One of the biggest risks for any hedge fund is that its human assets walk out of the door each day. Computers can absorb and react to information on a 100,000 securities instantly unlike a human trader. They can analyse new data, have the order in and executed before a human brain has even noticed the information. Computers don't think they are a genius when they fluke a lucky trade. They don't take lunch or vacations. They don't quit and try to set up their own fund with proprietary information. They don't have clandestine meetings with competitors. They don't complain about colleagues, clients, salaries and bonuses. They don't get sick or crash their Porsches. They don't lose interest after the IPO. And once the programming and testing is done you have eliminated key-person risk. There is a lot in favor of purely systematic strategies IF they are good.
Few investment managers admit to using that big institutional no-no called technical analysis despite the fact that many do. But calling it quantitative analysis is still ok, just. Computing power allows detection of predictive patterns and structure; we've long moved on from public domain moving averages, breakouts, candlesticks, RSI and MACD. Technical analysts look at patterns of prices and volume while fundamental analysts look at patterns of earnings and book value. Growth investors are trend followers while value investors are basically countertrend. Are fundamental analysis and technical analysis that far apart or is it just a change of inputs to the model?
Even if you buy into the "normal" nonsense, 95% Var means that about 1 day every month on "average" you will lose more. While $480 million losses may look bad, on $10 billion notional it is only 4.8%. If the Morgan Stanley quants made the human decision to run $2 billion notional cash at 5x leverage, losses of that magnitude, while bad, are not completely beyond the realms of expectation. The valuation noise on large portfolios is going to be tens or hundreds of millions even in relatively quiet times let alone market stress. Strangely no heads have rolled, yet, at Goldman Sachs' Global Negative Alpha "hedge fund" despite squandering over $3 billion of client money on its disastrous factor "models". $8.4 billion losses, mostly due to buying market share in CDOs and structured credit with little concept of risk, are another matter. The Merrill Lynch losses were due to human decisions.
Computers are at the mercy of what data their programmers choose to give them. Even genetic algorithms and neural nets rely on the system parameters and data sets provided by humans. Computers have solved simple finite systems like a chess game because it is a closed and rational problem. There is always an optimal move in any situation. But financial markets are much more complex, require decision making under uncertainty and the rules change while you are playing the game. We are a long time away from artificially matching the kinaesthetic intelligence of a basketball or soccer player. The intelligence necessary to master ball games is far beyond that required for board games. In financial markets computers are just a basic aid to human decision-making.
Email this • Subscribe by email • Google Bookmark This • Share on Facebook
© posted by Veryan Allen at 10/24/2007 06:05:00 AM
No comments:
Post a Comment