Hacker News new | past | comments | ask | show | jobs | submit login
Why didn’t people in finance pay attention to Benoit Mandelbrot? (reuters.com)
86 points by cwan on Oct 19, 2010 | hide | past | favorite | 48 comments



Because what his theory boils down to is that you can't predict the market. Pointing out that the market is describable by a self-similar fractal froth is an interesting thought, but it basically means that if true, you can't predict anything with any effectiveness. (Oh, you might be able to use that idea to tune yourself up something that might work slightly better on a small time scale, but as the market has sped up that small time scale has gotten awfully small....)

Who wants to listen to a party pooper like that?

I've found a similar problem at work. It is my belief after studying the problem for years, reading the work of many other of our smartest people trying to solve it, and over a decade of experience, that estimating software times are simply impossible except in the grossest of terms on any significantly-sized project, especially as you get into the multi-month estimates. Once you accept that, you can actually deal with it; Agile is in some sense a response to this problem. But try convincing other people of that fact. They'll tell you marketing needs some idea of when features go out, that management needs some ability to plan on things, that sales needs some concept of when to sell features. Well, too bad! None of that makes it possible. That is simply an extended explanation of why it sucks that we can't have these accurate estimates. And yeah, I'll play the game because it's still better than nothing.

I'm somewhat less sure that's true in the financial world. Shall we say, the evidence somewhat suggests that pervasive underestimation of risk can potentially have slightly negative effects on the global economy.


Who wants to listen to a party pooper like that?

Yes, yes, yes! The exact correct point.

Mandlebrot's finance papers, in published in the 1960's, refute Black and Schole's Nobel Prize winning theories of the 1990's. Who wants to hear that?

Does Mandlebrot rate a Nobel himself? Why of course not...

Mandlebrot pointed the way to "another country", one which marketing can not describe...


Actually, I thought I read about someone who bought a certain position in the market such that, if everything was priced correctly, they would lose a known amount of money each day.

But every time a "black swan" sort of event occurs, they'd make big money. It was on TV, so I don't have a citation handy, but I suppose that they could be said to be making money off of the assumption that people can't correctly price these things.


That would be Taleb & Universa (the true extent of whose success has been questioned, fwiw)


I'd like to know that position. Because then I could try to invert it (i.e. turn longs into shorts and vice versa).


If you'd searched for "Black Swan" it would have told you all you need to know.

Betting $1 at 1,000,000:1 that today something unlikely will happen is pretty much the same as playing the lottery.


I think the standard model also tells you that you cannot do much, no ? The efficient market hypothesis from Fama somewhat boils down to the fact that you cannot beat the market unless you have information that other don't have.

Another reason for the "conventional" methods success seems to be related to their empirical testability, at a certain period of time. The best article on Mandelbrot and the link with finance I have seen so far is on econoclaste (http://econoclaste.org.free.fr/dotclear/index.php/?2010/10/1...), but in French unfortunately.


I think the standard model also tells you that you cannot do much, no?

Well, as a matter of fact...

--- No, not true.

The "standard model" predicts a kind of randomness which is fundamentally tractable. It comes down to Gaussian versus non-Gaussian stable distribitions. A Gaussian model predicts that total market changes mostly come from day-to-day, small incremental changes - ie, a Gaussian model is equivalent to Brownian.

In stable, non-Gaussian distribution, a good percentage of total changes come from a finite number of rather large changes.

In the Gaussian model, an investor has time get out before the going gets rough. In a non-Gaussian model, that investor doesn't. The last few years have made the non-Gaussian/L2-stable/"Mandlebrotian" model much more plausible.


No, he was right. The standard model does tell you you can't make money. You are addressing a completely separate issue (hedging risk) in the rest of your post.


If by the "standard model", one means the general model of Black-Scholes and modern mainstream mathematical finance, I stand by my original claim.

In practical terms, modern mathematical finance's modeling of market with the Gaussian distribution has the implication that it is possible to add together a number of risky items to get a product which is less risky. This was the basis of Long Term Capital Management and this was the basis of the "synthetic" Triple A bonds built out of sub-prime mortgages.

The "free money" comes of out the risk combining/hedging approach through the implication that by adding up supposedly uncorrelated risks, you can create a lower-risk financial "vehicle" that still delivers a rate of return somewhat comparable to underlying items. If a bank pays 1% interest and you can get a 5% "virtually risk free return", then you've got 4% profit. Now Black and Merton of LTCM went one step further. They were so mathematically impressive that they got a virtually infinite line of credit to borrow against to use in the risk-combining approach. The implication was they would be, again, getting a nearly-risk-free rate of return and so investment banks could lend to them nearly-risk-free too. Even their massive failure didn't convince people. The entire stable of CDO etc product sold on the same basis, the basis of providing risks no greater than the highest rated corporate bond but with significantly higher return. We can see how "risk-free" they really were.

Moreover, it is true that if you can find a bunch of small, uncorrelated risks with finite mean and variance, you can add them up to get a big, tractable Gaussian distribution with a very small variance (that's the mean-value theorem, in fact). So the synthetic bond approach rests firmed on the standard modeling of the market as akin to Brownian motion.

The problem with this approach is that, as Mandlebrot pointed out is that one doesn't actually wind-up dealing with distributions having finite variance. Adding up distributions without finite variance gets you a distribution in the L-stable family of distributions, which are in general much less tractable, not having finite variance themselves. In this light, it seems clearer why synthetic bonds turned not to be the free money they claimed to be.

One might argue that this mean that the "efficient market hypothesis" itself would imply that markets don't follow a Gaussian distribution. I'll leave that those who still some faith this formulation - I'd personally claim this "hypothesis" isn't even a coherently position.


In practical terms, modern mathematical finance's modeling of market with the Gaussian distribution has the implication that it is possible to add together a number of risky items to get a product which is less risky.

Nonsense. Any Finance 101 textbook will tell you this is only true if the securities are uncorrelated. It will also tell you that assuming the EMH holds, people will buy into the lower risk index until returns are reduced to 1%.

As for LTCM, the thing that killed them is that they didn't have an unlimited supply of credit and they got hit by margin calls.

The entire stable of CDO etc product sold on the same basis, the basis of providing risks no greater than the highest rated corporate bond but with significantly higher return.

No, the entire stable of mortgage-backed products was backed by the assumption that housing won't go down crash, a factor exogenous to Black Scholes. Also, you seem to not understand how CDOs are priced - the Black Scholes style models are used for interest rates, which have not exhibited infinite variance. Default rates, which are included separately, also have not exhibited infinite variance (in fact, since they are bounded between 0 and 1 they cannot exhibit infinite variance).

No model works if you plug in the wrong parameters. To quote Babbage: ...I have been asked, – "...if you put into the machine wrong figures, will the right answers come out?"... I am not able rightly to apprehend the kind of confusion of ideas that could provoke such a question.


The determining factor in economics varies as a function of the problems and matters connected to what is inside people themselves. "Because the consciousness that moves us is making a world."

A simple example to demonstrate that it is possible to predict things about this society to a nontrivial degree, is the products and activities of Apple Inc. They have great marketing foresight. But besides that, it sounds like they were building mobile devices for a long time before the touch-based smartphone market really took off in the last few years, enough to get it right. I'm not an Apple employee but I've been using an apple for ages. It seems to me like the people at the group had to very carefully understand what kind of questions there are in building something like that and making it work best. Granted, Apple has lots of problems, but what I'm trying to point out here is that by funding and working on the problems in order to get it right, is usually going to result in the best products which if able to be made available widely enough will become very popular for a period of time. As a result, objective-C has come into use much more, and jobs were opened.

The question of how to see and predict the course of the world lies in understanding what exists in front of us.

Products that were developed by copying others or with ignorance of the problems in the matter, will eventually exhaust themselves. On the other hand, products from a company that had see what exists in their life and generates work matching what they see in problems, assuming they are adept at protecting themselves in business, will be able to be more successful. Even people who don't have any university education can be successful through their knowledge of what is. So we have to be able to comprehend human consciousness in order to understand how an economy will move. Luckily, to understand consciousness is simple, if you are really willing to know. Consciousness is changed for better or worse by what happened to it.

In this society where people have undertaken large-scale industrial production on Earth, people in presently affluent circumstances can afford to use cell phones more profitably than without them. But in a different society, the problem that makes the economy exist may be different but the principle through which it exists remains the same. Things are either getting better or worse through what is in the principle.

This principle isn't precisely comprehended in those economists' education and that results in lots of thinking that turn out to be misguided. But things in reality are very simple. It's just that we need to search what kind of problems we have inside of us in order to know what they are like.


You are right that predicting trends is not that hard. When it comes to financial prices the issue is that you are effectively predicting small changes in trends, plus a lot of noise added on by trading activity itself. Much harder.


"Supercomputers" here we come... :)


For those who don't understand the Black-Scholes references, here is a quick explanation.

What Black and Scholes proved in their famous paper is that if you start with a portfolio containing a certain fraction of money and stock, and rebalance the portfolio constantly as the stock price moves according to certain rules, then after a fixed amount of fluctuation in the stock price you will be left with either pure stock or pure money, depending on the eventual price of the stock. In short, this portfolio acts exactly like an option. And therefore the price of an option should match the price of the portfolio. If it doesn't match, then you can buy one and sell the other to get free money until they do match.

This is fine, but in the real world we trade options based on a fixed period of time, not a fixed amount of variation in the stock. The answer to this which finance uses is to estimate the volatility of the stock. Given known volatility, and a known time period, you can tell how much variation there is, and then use the Black-Scholes model to price options.

Mandelbrot's critique of this is that volatility itself is not constant. Therefore you can't really predict when the Black-Scholes portfolio will expire. In particular the model systemically underestimates the likelihood of extreme events. When this catches all of finance off guard, the result is frequently some sort of crisis.


My suggestion why people didn't pay attention? He had no real theory. I've read a couple of his books, here is the general formula: "Look, gaussian's don't fit the market well. Aha, I've made a graph that looks visually correct!"

Now, it's absolutely true that gaussian's don't fit the market well - they only work sometimes. The thing is, everyone knows this, and tacks on additional features to explain the other phenomena. For instance, one might assume movements are normally distributed, except for short term spikes followed by high volatility. A good risk manager will throw non-gaussian volatility at a model during backtesting.

Mandelbrot's fractal story just didn't add much. The black scholes story has some convincing theoretical background (it assumes an "evil" market out to get you [1]). It misses things, but many of the things it misses can be added in, in a more or less convincing way - e.g., I understand the black scholes part of my model handles small movements, and the stochastic jumps handle the big ones.

All Mandelbrot's model gives me is a graph that kind of looks right - it doesn't give me any understanding.

[1] This is not the textbook description, but Bob Kohn convinced me this is the best way to think of it.


What if the understanding you have is fundamentally wrong? That's the point - he wasn't saying that he had a replacement for Black-Scholes. He was saying that anything based on Gaussians is broken beyond any utility, no matter how tempting it is to believe the theoretical story behind it. You don't need to be a tailor to see that the emperor has no clothes.

That's the tragedy, in a way. The portfolio managers were saying that they needed a tool to assess risk, and that any tool was better than nothing. Mandelbrot turned around and proved that a bad tool is worse than nothing, and got ignored because it might take another hundred years of research before his ideas can get turned into a practical tool.


This would be fine, except that the short, rare, unpredictable spikes are overall responsible for as much variation in value as the "predictable" movements, which more or less makes all the exercise quite pointless.


How does the fact that a model has two features make understanding one of them pointless?

If you are seeking a grand unified theory of everything, maybe it's pointless. Black Scholes is just a model. If you are seeking to make money, Black Scholes is one tool that can help you do that. And make no mistake - people do make billions trading models which incorporate elements of Black Scholes.


They make billions for some time, then all their profits are wiped out by some "black swan". Fortunately, our governments are here to save the day...


They didn't ignore him. Fat tails and jumps are just really hard to estimate in finance. More interesting is that the unstated premise for these link-bait titles (aka great headlines) is "If only they had listened to Mandelbrot, we could have avoided all these problems."

It is nearly absurd on the face of it, but worth a comment. Like Louis Bachelier's description of stochastic processes in 1900, mathematicians like Mandelbrot have been inspired by financial time series to develop formal descriptions of the phenomena. Sometimes, this leads to a marginally deeper understanding of the object of study.

Regardless, mathematicians and other scientists have little interest in developing prescriptions that would help control or reduce volatility. Most observers seem to think that the amount of financial volatility is way too high in comparison to the underlying economic realities. A few, like Fisher Black, have had the opposite opinion and suggested that the prospects of the underlying economy (including human capital) are actually hugely volatile. Virtually all take the structure of markets as a given and assume they are low-friction and generally structured well.

My own opinion is that volatility is mainly a function of information starvation in the market. Just look at the poor quality of financial accounting, auditing, and the cherished secrecy of large risky positions and you can see possible areas to unlock information flow that would allow markets to do a better job of tracking "true" value and ignoring chaff generated by the act of trading.


Or just look at banks.

If we did not have deposit insurance from the state, banks would probably be much more pro-active in disclosing all their positions (including liabilities), so that people still trust them. Some banks showed their liabilities in the crisis, but this was way too late for people to pick them apart, so it did not help with the trust.


Excellent point. Instead we trust the regulators to make sure banks are a-ok. We would do well to "crowdsource" bank regulation by requiring banks to stream their positions to the public.


Yes, and just don't offer deposit insurance, so that the public scrutiny has some bite. Similar to how the `bond market vigilantes" the Economist is so fond of keep the yield on government debt in line with the governments' fiscal and monetary policies.


I used to explain this with Upton Sinclare's great quote: "It is difficult to get a man to understand something, when his salary depends upon his not understanding it!"

Now, older and perhaps a bit more jaded, I would modify that to "Do not assume that someone doesn't understand something. Assume that he is assuming someone else doesn't understand that something."

Or, more succinctly, "There's a sucker born every minute."


Sinclare's quotation describes a self-actuated internal lack of awareness while yours does not.


Call it personal growth.


The most important take-away from Mandelbrot / fractals as it applies to finance should be the realization that fractals can represent better ways of presenting or simulating financial data than brownian motion / random walks / Black-Scholes. As to why this hasn't been accepted more broadly - well, as the article briefly mentions, there are powerful individual incentives for people to continue to play along in the charade.

If this has not been proved EXTREMELY WELL by events in recent history, I don't know when it would be - but whether from LTCM, or more recently seeing so many CDS etc blow up, it is obvious that many "once in a million" probability events exist than are considered in a proper normal distribution.

Mandelbrot was once asked whether he had any particularly successful strategies for dealing with the market. He said, well, I don't discuss those things - because if I was correct, everyone would follow my lead, and the strategies would no longer work; and if I was wrong, people would discredit the thinking behind it!


...but whether from LTCM, or more recently seeing so many CDS etc blow up,...

I'm confused - how do CDS (did you mean CDOs) blowing up prove that a fractal model of the market is better than the standard Black Scholes + assorted tweaks model?


Models with constant-volatility random walks aren't favoured over "fractal" models because of some elaborate charade or flawed incentives. They are used because they are tractable models that can be used to make predictions. Fractal models are not.

Financial models are just like any other engineering tools. They approximate reality so they can be useful; but violate their assumptions or use them outside of their intended purpose, and they're likely to blow up in your face.


One of them did -- Nassim "Black Swan" Taleb was great friends with Mandelbrot.


Mandelbrot had qualitative and quantitative insight, but I am not sure Nassim applied fractal brownian motion modeling/simulation to position himself for tail events, but more simply had the qualitative understanding of where tail event risks were greatly under-priced in the system at the time before anyone else.

In terms of fractal brownian motion vs black-scholes-merton, it is a question of practicality. It is really easy to hedge very complex portfolios with large positions using BSM, especially after adding a few considerations to extreme possibilities in volatility. Without BSM, we'd still be in the dark ages with Option Seller(Writer) firms scalping buyers with option prices 10 times higher inflation-adjusted than today. Mandelbrot doesn't offer a practical alternative, and this is the part that people didn't listen to Mandelbrot on, but that doesn't mean Mandelbrot is not awesome.


Recommended Mandlebrot read: 'Fractals and Scaling In Finance' from 1997. http://www.amazon.com/Fractals-Scaling-Finance-Benoit-Mandel...


Financial markets, from the perspective of mathematics, are quite interesting.

I wrote a paper that referenced some of Mandelbrot's work, namely that cotton markets followed a Levy stable distribution:

http://en.wikipedia.org/wiki/L%C3%A9vy_distribution

It's an interesting distribution as it pops up in nature a lot too, such as how birds find food.

Mandelbrot's work and the work of many others have shown the movement in security prices resemble random walks. There is an awful lot of work that shows markets are basically unpredictable.

Yet the reality is otherwise. People make an awful lot of money by predicting markets, which shouldn't really be true to the degree that it is if movements were random. There are trends like Mondays tend to be down days, Fridays tend to be up days and so on.

Probably the biggest failing of financial modelling is the failure to adequately factor in the fat tail. Incredibly unlikely events tend to be more common than otherwise modelled. The collapse in the subprime mortgage market is the most significant recent example of this.

Warren Buffett once characterized how many traders operate as picking up pennies in front of an oncoming bulldozer. The vast majority of the time it's safe but it's not 100% safe and the consequences are completely disproportionate to the reward. It's a challenge to model that kind of scenario.

Anyway, RIP Benoit Mandelbrot. You were a mathematical visionary.


Benoit wrote a scathing critique of modern economic theory and the assumptions upon which it rests. Attacks on fundamental belief systems are generally impolitic.


Here are a few pdf's of some articles in Wilmott Magazine touching on this issue, and Mandelbrot's work in general:

http://www.wilmott.com/detail.cfm?articleID=342

From the Emperor With No Clothes article:

And so why, we are obliged to ask, has one of the most important discoveries in the history of economics failed to inspire a concerted effort to develop a better theory? Perhaps it has to do with Mandelbrot himself and his position vis à vis the kingmakers of economics who reserve the right to bestow acknowledgment. Why has Mandelbrot not been recognized, say, with a Nobel Prize in economics? Because he is considered an “outsider”—trained as a mathematician and active in research that ranges well beyond economics alone? Or has he failed to play by establishment rules and violated some unwritten code of economist conduct?

Why, indeed, should Mandelbrot receive a Nobel prize in economics? The Nobel is the ne plus ultra of global recognition; it carries considerable political weight; it does not go unnoticed; the very fact of the award can stir things up. In Mandelbrot’s case, everyone would be made aware that classical economics – quite literally the emperor of our global economy – is without clothes. Younger economists and students around the world would be motivated to search for a better explanation of how economies work and why, and to propose alternative theories validated by actual data and subject to empirical scrutiny. Putting all of us, as real science always does, in the temporarily uncomfortable position of not knowing in order that we may know.


I really wouldn't take an article seriously if it partially blames the '87 stock market crash on Black-Scholes (basic model for pricing European options). That makes literally zero sense.


It makes literally zero sense unless you know something about that crash and the financial markets of the time. In which case it makes perfect sense.

The Black-Scholes model provides a recipe for creating synthetic options that will (under the assumption of known volatility) act just like real ones. Which is convenient because you can create synthetic versions of options that people want to have but which are not traded. Leading up to the '87 market crash, lots and lots of these synthetic options were created. Then came the crash.

People can debate endlessly about why the crash started. But once it did, there is no question that trading algorithms attempted to close out trades that were necessary to maintain synthetic options. These large trades attempted to execute in markets that had seized up, and made the market much, much worse. The result contributed greatly to the crash, and caused the synthetic options to fail to work as promised. (Besides, the Black-Scholes algorithm guarantees that it acts like an option through a certain amount of variation in the stock price, and not for a particular time period. The volatility of the crash demonstrated the importance of this discrepancy.)

Now do you see how the Black-Scholes model contributed to that crash?


automatic trading based on Black-Scholes acerbated the crash != the Black Scholes model contributed to the crash

Unrelatedly, is there anything wrong with, say, a mortgage-backed security accurately priced according to a particular model, so long as the limitations of the particular model's assumptions are properly understood? This applies to any model in economics or finance. All are obviously just simplifications of reality.

The technical assumptions of a mathematical model should not be blamed for the actions of ignorant or reckless investors.


You don't get a very big explosion with just a fuse, and no combustible material. Without the combustible material, the fuse just fizzles out, no great harm done.

It's a separate thing to blame a bad idea, versus blaming the people who thought the bad idea was true. Language is ambiguous; trying to weasel one's way out of "this idea is bad" by saying "the people who think this idea, they're bad; it's not the idea itself", is IMO trying to rely on the imprecision of casual language to refute an argument only for a single formulation, but not in spirit. Ideas have no life of their own outside people's heads. The same argument can be applied to say that there is no such thing as a bad idea.


The problem with trying to use Black-Scholes only within its limitations is that it would never, ever get used in any real market. Its fundamental assumptions just don't match the way price movements actually play out.


No kidding, blaming the math for the crash is like rearranging deck chairs on the titanic.

"Our models told us our plans were unsinkable so we went right ahead into the icebergs"


Exactly.


Found this quote interesting.

---

Black-Scholes and the many financial risk models that have evolved from it (including Felix’s friend the Gaussian copula) are all about volatility being measurable and predictable. “When Black-Scholes came out, I said, ‘Well, it won’t last,’” he told me in 2005. “‘I’ll come back when it’s gone.’”


Oh, a lot of them did. Many of those can only be described as religious cult followers - practitioners of Elliott Wave "Theory".

Some even went as far as accusing Mandelbrot of plagiarizing Elliott's work: http://www.math.utah.edu/vigre/reu/reports/harris_fall2005.p...


I've read some of Mandelbrot's works on finance and I would never put him in the same category as the "technicians". It's hocus pocus mumbo jumbo that I don't think he would subscribe to (e.g. Elliot Wave's "5 up, 3 down" principle).


Agreed. And finance was never the primary objective of Mandelbrot's development of fractals anyway, just a side effect of varying degrees of interest.

Elliot Wave appears to be an attempt to back fit a rather simplistic theory to patterns in financial time series data. Fractals are so much more.


Money talks so saying "it will never work" to people rolling in money won't have any effect.

Had Benoit raked in the dough, it would be called "The Mandlebrot Strategy" and everyone would be using it.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: