I want to shed some light on this topic. As a former HFT quant, I find the general "Main Street" sentiment toward the profession to be misguided.
1. HFT does NOT take anything from most investors most of the time. Most investors trade on a much longer investment horizon than any HFT firm. If you are a normal investor, you can lump up most of HFTs into the same bucket as the exchange itself. Most of them engage in some combinations of (i) pure arbitrage (ii) very short-termed statistical arbitrage (iii) market-making based on (i) and (ii). None of this is really that relevant to most investors. The only time HFTs can screw over a lot of investors is when their software goes awry. But this is a risk that any computerized system has, high frequency or not. If there is a major bug in Chicago Mercantile Exchange's matching engines, that would be a total disaster.
2. HFT firms do have huge execution risks: if you are making a two-sided market, there is a chance you can get "swept", meaning when the value of the underlying moves faster than you can react, you get filled on one side of the order without a realistic chance of hedging for a profit. HFT firms do a ton of research into estimating their execution risks. (edit: This point is often glossed over, making HFTs look like this evil superpower group of nerds exploiting other investors. That's not the case)
3. Finally, most HFT traders don't go into the profession solely for the money, just like most people do not apply for YC solely for their passion. Most of my former coworkers weren't that greedy and led pretty modest lives despite making hundreds of thousands of dollars. For them, high frequency trading had a locally optimal balance of tackling intellectually challenging problems while getting paid handsomely.
I won't say HFT is the most valuable thing that its practitioners can be doing. I think the (trading) world would operate just fine without all these micro-second level transactions. But they are NOT the next subprime mortgage, and I just hope folks stop commenting on stuff they have no clue about.
> I find the general "Main Street" sentiment toward the profession to be misguided.
> I won't say HFT is the most valuable thing that its practitioners can be doing. I think the (trading) world would operate just fine without all these micro-second level transactions. But they are NOT the next subprime mortgage, and I just hope folks stop commenting on stuff they have no clue about.
Your post is interesting and I learned from it. However you admit that HFT is not the most valuable thing smart people could be doing - this is the reason that many dislike it. HFT provides little value to humanity or society. I do not think this makes me 'misguided' or 'have no clue' and I don't like how this makes it okay for advocates to sweep away concerns about it. We could easily say the same thing: algorithmic trading in itself is misguided: it does not use technology to solve socially useful problems such as global distribution, food production or folding proteins to better understand drug interaction and find cures for disease.
I think we can say similar things about Google engineers working on making infinitesimally small ad targeting efficiency gains, yet this group typically has much less vitriol directed at them, though the degree to which this is the case may differ.
The counter argument to this would be that said efforts help fund more genuinely beneficial work such as their recent work in robotics and proliferation of internet availability globally, but how the net balance of talent productivity plays out is murky. The ad team is a large, multifaceted team inside Google.
Difference being engineers don't get paid as much – though perhaps they should be.
Finance guys are much more aggressive about capturing more of the revenue they create in salary and bonuses, while engineers seem to shrug at being "overpaid" while letting proportionally more of their value go to execs, investors and giant cash hoards.
I have not seen that to be the case. As a software developer in the HFT trade I make about the same as I would working for google. If the firms I work for take off or have particularly good years, I might make a lot more than the google salary bands, but it would be about the same as if I were an early employee (but not founder).
Now if you are talking about ibank traders who HFT are replacing, that is true. Their salary bands are enormously higher than engineer pay scales. Sort of explains why they direct so much vitriol towards HFT.
In general though, finance firms (at least the publicly traded ones where we can see their books) reserve a higher proportion of revenue for compensation compared to tech firms. Not entirely sure at HFT shops, but I think that may be what the grandparent comment may be referring to (and I agree that we could do better if our goal were to be more equitable -- but no one has an incentive to do this on a grand scale yet)
That's the thing. There are only a couple of publicaly traded HFT firms and they are tiny. Their compensation looks more like a technology firm than a "finance" firm.
Remember, that the vast majority of people who make large compensation in a finance firm are essentially sales force. They are selling complex products, financial services, or operations.
HFT of type #1 means that you can go to any market and roughly get the same deal.
IE: You can purchase a stock in Euros in some European exchange, OR you can purchase a stock in Dollars on the NYSE. In both cases, you get the same value.
This equivalence in value only exists because traders perform arbitrage on fractions of a penny over the course of milliseconds. And thanks to HFTs, any divergence in value larger than a penny or so is arbitraged away within miliseconds.
This is a solid service for all involved, and HFTs are the ones who take the risks behind arbitrage. (IE: if the markets move in an unpredictable way)
Many "Main Street" investors have their retirement savings locked up in large funds that trade much more frequently than the investor themselves do. Would you not say that HFT is skimming off of those trades, and ultimately skimming from the "Main Street" investor's retirement savings? Would you say that the marginal value that these firms add for those investors' funds is worth the premium?
Is it clear that the reduction in bid-ask spread is due to HFT rather than some other change in market dynamics? I frequently see "Bid-ask spread has reduced by x in the last two decades, therefore, HFT is good", but I've never seen a causal link explained, and a lot of other things have happened to the structure of the markets in that time.
I think it's pretty clear that HFT has a big role in this. If you look at the order books of US equities, most of the most aggressive orders on the book are ones that look like they come from high frequency strategies. Furthermore, if you believe that around 50% of trading volume is from HFT, and most of them are market making, then yeah, it lends one to conclude that the high frequency players are the ones who place more aggressive orders (which means, ones which reduce the spread) which gets them executions.
It's not exactly proven and would be very hard to prove. That said, nearly everyone thinks electronic markets are better than the alternative. Any electronic market will open itself to algorithmic trading, so what most people are actually worried about are unintended consequences of changes to the current system.
That and for most people who actually understand the mechanics what is going on, the current system seems way better for the vast majority of people than the one it replaced, so we have little reason to change.
The industry doesn't really make that much money. Estimates for this past year were in the $1-2 billion range. That's not big for the finance sector, or any industry. That being said, it doesn't take that many people to run a shop, so individuals may be paid a lot.
They don't make tons of money. I think Michael Lewis quotes a annual revenue of 16 billions for all the HFT companies put together. That's less then half of goldman sachs' revenue alone.
That's arguable. I never had a chance to estimate this in earnest, but I have the hypothesis that the majority of HFT transactions are with one another. Sure, they collectively provide liquidity to non-HFT investors (because the order flow has to come from somewhere), but I also had the hypothesis that if the clearing houses (CME/NYSE/etc.) could improve their matching engines and/or consolidate disparate exchanges, that might solve the liquidity problem without HFTs.
Also curious on your perspective of their role in market making. Do HFTs ever end up buying a larger block than they know is being demanded and end up holding some of the remaining for a time?
> What Lewis’s book demonstrated to me isn’t just how “bad” HFTs are per se, but rather, what happens when finance keeps walking down the path it seems to be set on — a path that involves abstracting itself from the creation of real-world value. The final destination? It will enter a world entirely of its own — a world in which it is fighting to capture value that is completely independent of whether any is created in the first place.
I'd say the financial markets have been there for 6-8 years already.
Nothing will change, since it's turtles all the way down.
The first person to discount a bill at the very first bank was "abstracting itself from the creation of real-world value".
Later, I'm sure someone said the exact same sentence that you just did when the first exchange traded fund (ETF) was created. I know for a fact it was said throughout the 90s in relation to derivatives, etc.
In the future, HFT as it exists today will be a quaint feature of a bygone era that will be referenced to illustrate the problems with whatever new practice has superseded it.
What does matter is where you are on the spectrum and how fragile that makes you, but there is no natural law limiting how abstracted you can be and how fragile you can be. By some relative measures we're impossibly abstracted and fragile, but by other relative measures (as yet unknown) we're adorably simple.
I thought EFTs were only possible due to HFT, which seemed like one of the greatest benefits to retail investors. A little bit of arbitrage tax is no where near as bad as sales loads and management fees.
The only problem is that arbitrage tax is hard to measure, while sales loads and management fees are predictable values. But I agree with your point overall.
Getting pennies skimmed off of my trades is not nearly as big a deal as a 0.1% yearly recurring fee on say... a $100,000 retirement account. (ie: $100 / year). Those tiny percentages add up to a huge amount of money in the long run.
I've heard of reports of 1.5% or even higher management fees (or ~$1,500+ / year in this example)
HFT provides liquidity, and extracts a price for providing that value.
HFT seems like stealing only because they are using advanced technology methods to make money, whereas in the 1980s and before market makers routinely manipulated the spread and pocketed likely a similar percent of trading profits.
At least the online brokerages and pioneers of HFT broke down the antiquated 1/8 stock price ticks and lowered the spread and per share transaction costs.
Market makers had two jobs. Provide liquidity, and provide price stability. HFT does a better job than they did at providing liquidity, for a lower premium. However it does a worse job at providing price stability.
Furthermore to me HFT looks like stealing because it is very inefficient relative to the need. If exchanges were to reorganize how they work slightly, we would get much better price stability, somewhat less liquidity, and have a much lower premium.
The necessary change is simply this. Exchanges would have to set a price at which orders are to execute. Whenever there are excess orders in one direction or the other, that price would drift up or down, slowly, until the order executes. The result is that trades might take a minute to execute, but margins would be far less. Sure, a ton of finance people would jump up and down screaming bloody murder about how many of their complex trades would be affected. But my opinion is that that economic activity takes a lot of brains and resources, and doesn't significantly add to the purported purpose of finance - to make money available to companies engaged in useful production.
> The necessary change is simply this. Exchanges would have to set a price at which orders are to execute. Whenever there are excess orders in one direction or the other, that price would drift up or down, slowly, until the order executes.
How would this even work? Understanding Market micro structure is literally my day job and I'm having trouble parsing your proposal.
Consider the market where I want to buy at a limit of 9.90 and someone else wants to sell down to a limit of 10.00.
What price would the exchange set? People have already said the maximum the market will buy for is 9.9 and the minimum the market will sell for is 10.
if there is no overlap you just cant' trade no matter what the exchange does.
What I am describing is not how markets work today, so you can't understand it by thinking in terms of how markets work today.
Today the price is the price at which the last transaction happened.
I am proposing instead that the price is set and updated by the exchange, and transactions only happen when they can execute at the exchange price. And the exchange price moves at a preset rate - down if there are outstanding sell orders, up if there are outstanding buy orders, else still.
So in your situation the price would wind up somewhere between 9.9 and 10, then stay indefinitely until another order began moving the price.
Basically for someone placing an order you're losing the ability to trade instantly, and gaining the prospect of trading at a possibly better price. Therefore you don't need a pool of people trying to give you instant trades.
The HFT sees that there is a 10 cent gap and decides to put 2 new orders into the market, 1 sell at 9.99 and another buy at 9.91.
Now the spread is 8 cents. Anyone who wanted to buy can now buy it at a penny cheaper than an instant ago and the opposite holds true for anyone wanting to sell.
> However it does a worse job at providing price stability.
Isn't this only true if you measure price stability based on amount of changes in the price, but not magnitude of those changes? How can they lower the spread but simultaneously decrease stability?
Yes. You get liquidity, meaning more investment is available for some companies - people who would have not invested now will invest. The spread is lower, benefiting buyers and sellers. Work is automated meaning market makers do not have to stand around on the NYSE or CBOE all day - people can sit in their offices, thinking up algorithms, backtesting them etc. (of course the quants doing this are not the same people who had been doing it usually).
The old time finance guys have no idea what a back-propagating multilayer neural network is and how it's eating their lunch, and there is resentment and suspicions of foul play. I am more suspicious about the suspicion. You can pitch a tent and set up a hedge fund like Renassiance or DE Shaw without the blessing of an investment bank, and perhaps the banks resent this, despite the fact that they're doing HFT as well.
I'm not an expert here by any means but I think HFT seems like stealing to most of us because it is largely hidden from public view and from any serious scrutiny. The price it extracts is largely unknown and not well understood by those who are paying it. The biggest HFTs seems like a shadow government running the markets and extracting a hidden trade tax.
As for the argument that the value they provide is "liquidity", that also seems dubious. I've seen that argument thrown around a few times now, but I'm not entirely convinced that HFTs, as they exist and work now, are achieving this goal fairly, nor that this is necessarily the best solution.
Do you think that pre-HFT trading practices are transparent and easily understood, and that they're under a proper amount of scrutiny? Are large banks trading at lower frequencies less shadow governmenty? I posit they are not, and I don't understand why the speed of the transactions would be dispositive.
(I'm told by knowledgeable people that there are indeed some shady things that happen in HFT, such as quote stuffing, that ought to be fixed, which makes sense. But I don't think that brings us anywhere near "shadow government", nor does it seem like a damning problem for HFT as a concept.)
The odd-eighth spread pricing scandal was also something that was hidden from public view for decades, and it had a far more profound impact on the fairness of the markets!
Unless you can say that the "price extracted" is fair and self regulating without any clear and transparent oversight, I think there is probably room for improvement and a benefit from increased scrutiny. However, as I said, my knowledge of the internal workings of the markets is still pretty limited.
If you understand the scandal, you understand the answer to that question: automated trading, opaque as it is, strangled the spreads that human market makers had been capitalizing on.
But how much more liquidity would it provide versus frequent batch auctions where trades are grouped in 1 sec (or maybe 100 ms) intervals and matched? Batch auctions would entirely remove the benefits of the HFT arms race (and other gaming like flashing a price and cancelling instantly)
Batch auctions do not deal with the central problem which is tie-breakers. What happens if there are more participants on one side of a price level than the other? What mechanism do you use to determine ties?
"There are some subtleties involved [...] since
it takes light roughly 4 milliseconds to travel between Chicago (where ES trades) and New York (where SPY
trades), [...] We would also like to suggest that the fact that special relativity plays a role in
these calculations is support for frequent batch auctions." (emph added)
Read Flash Boys, lots of HFT firms are exploiting the system, and they aren't adding any liquidity. They are using their speed to discover who wants to buy at what prcice, then they buy stocks faster then the other participant, to then sell it higher to that participant.
Don't read Flash Boys. Read Dark Pools. Dark Pools is also critical of HFT, but doesn't have the baggage that Flash Boys does. If all you know about HFT comes from Lewis' book, you should be concerned about having the whole picture.
In particular: you can come away from Lewis' book believing that the markets prior to electronic trading were reasonable fair, when the reality is the opposite: the trendline of fairness and transparency in the market is sharply positive, and the inflection point of that trendline is the advent of fast electronic trading.
But who needs the liquidity anyway? Seems like a "real" investor would buy e.g. shares for the expected dividends/profit after carefully studying the books, or for strategic business related reasons. Both buyers neither need to buy this very split-second nor do they care about the smallest price deviations.
Remove the liquidity (which comes at quite a steep price it seems), now what? There seems to be plenty of capital looking for investments.
The immense effort spent on HFT could now be concentrated on
a) finding things worth investing and finding better ways to do so
b) create more things worth investing in
The people who need the liquidity are the people who willingly take the liquidity. Do you think someone puts a gun to their head and forces them to submit a market order?
It's a little bit like saying the guy who runs your 7-11 is a scumbag because he charges an extra 50c for a carton of milk, and "who needs that liquidity, they could just wait until <insert large supermarket chain> opens and get their milk there". Sure, but they might want the milk now. And no one forces them to get it from the 7-11. It's a free market.
First of all, this argument begs the question: why are some people so insistent on buying milk right now that they are building multi-billion-dollar computer systems and running special data lines with sub-millisecond latencies to do it? Needing milk so desperately that you can't wait until the grocery store opens in the morning is supposed to be an unusual condition, not the normal way of doing business.
But more importantly, HFT is not the same as liquidity. HFT only works in the first place because there are already market orders being placed; if no one but HFTs wanted to buy or sell stocks, no trades would take place. So the HFTs, by themselves, can't provide liquidity; they have to wait for someone else to do it, and then execute their trades to help narrow the bid-ask spreads.
I'll agree that there is some value to narrowing the bid-ask spreads; but is it enough value to justify the enormous effort and the enormous amount of human talent that goes into HFT, that could be doing lots of other valuable things? I don't think so; I think all that is a sign that the incentives in our society are screwed up.
Are you aware of the distionction between market orders and limit orders?
HFTs by themselves can provide liquidity (like any other investor), namely by entering a limit order in the order book. They can also by themselves narrow the bid-ask spread, namely by entering a limit order at a price more competitive than the current first limit.
Of course, someone else has to send a market order to the exchange for a trade to execute.
> Are you aware of the distinction between market orders and limit orders?
Yes.
> HFTs by themselves can provide liquidity (like any other investor)
Exactly: like any other investor. Meaning, the "high frequency" part of HFT is not what provides the liquidity. (I admit that I should have made that clearer in my previous post.)
Suppose an HFT enters a limit order that is more competitive than the current limit. Either some other investor would have been willing to trade at the same price (as the HFT's limit order), or not. If not, then it doesn't matter how fast the HFT places the limit order; it's going to increase liquidity in any case. So the "high frequency" part is not necessary; the key is that the HFT is an investor willing to trade at a certain price.
But if some other investor would have been willing to trade at the same price, then the only difference the HFT makes is speed: the liquidity gets added sooner than it otherwise would have. So the "high frequency" part doesn't change anything except who the profits from the trade go to.
The "high frequency" in HFT compared to human traders is a little like the "high fructose" in HFCS compared to table sugar. To wit: "high frequency" changes the relationship of HFT to other sell-side traders; it's what allows HFT shops to potentially outcompete human market-makers.
However, it's the category of sell-side traders in general that matters for liquidity.
You're using imprecise language here (for instance, referring to market makers as "investors"), but the clear implication is that you believe that normal, "buy-side" investors provide adequate liquidity. But they don't, and we know that, because the more competitive the sell-side gets, the lower spreads get.
> you believe that normal, "buy-side" investors provide adequate liquidity
Well, of course that depends on how you define "adequate". What is all this incessant trading of stocks for? What value does it provide? For example, if I'm an investor (a "real" one, not a market maker, to use less imprecise language) with a long time horizon, something like 30 years, because I'm saving for retirement, how does HFT make me better off?
One obvious benefit of "incessant trading of stocks" is that buy-and-hold investors can move in and out of a position for much less than they could in the 80s. The "incessant trading" reduces costs, which can be very important even to value investors when there is some extrinsic prompt to trading.
> buy-and-hold investors can move in and out of a position for much less than they could in the 80s
In other words, brokerage fees, or the equivalent, should be lower, so the overhead to execute a trade is lower. That has some value, yes, but not a lot, because a buy-and-hold investor, of course, doesn't execute many trades, so the overhead cost of executing trades is already pretty small for him.
Also, most "buy-and-hold" investors hold mutual funds, not stocks, which changes things. See below.
> costs...can be very important even to value investors when there is some extrinsic prompt to trading
Yes, but as I just noted, most "buy-and-hold" investors are holding mutual funds, not stocks, so they aren't paying the direct costs of stock trading anyway. When I want to rebalance my 401k, I don't trade stocks, I trade mutual fund shares, and they're all shares of different funds offered by Fidelity or Vanguard or whoever my 401k provider is. (And with most 401k's, certainly with the ones I have, as long as you don't rebalance too often there is no fee for rebalancing.)
So decreasing the overhead cost of trading will only appear to me, if at all, as a decrease in mutual fund fees; but with most 401k's the individual doesn't see those anyway, because they're provided through employers. I don't know how much the fee question affects the negotiations between employers and mutual fund providers for setting up 401k's, but in any case that's at least two layers of indirection between me as a retirement investor and the overhead cost of executing individual stock trades.
So I can see some small benefit, yes, but is it enough to offset the high social cost of having so many smart people doing HFT instead of something more productive?
Whatever you pay to your brokerage, you pay on top of the spread. The spread is what you pay to place a market order.
In exchange for the privilege of buying right now, you pay the best offer price. In exchange for the privilege of selling right now, you pay the best bid price.
It's a commission you pay per share; the more shares you try to move, the more you pay whoever's providing you the liquidity.
> Whatever you pay to your brokerage, you pay on top of the spread. The spread is what you pay to place a market order.
I agree with you for the case of a "buy and hold" investor buying or selling actual shares of stock (with the proviso that the overhead to such an investor is already pretty low since he doesn't execute many trades, so the gain to him from something like HFT decreasing the bid-ask spread is small). But as I noted, many, probably most, "buy and hold" investors hold mutual funds through 401k's, not individual stocks.
For example, I have a 401k. I don't place market orders. I rebalance my portfolio every so often, which, after a lot of intermediate steps, might result in some mutual fund manager placing a market order. But I don't see any of the direct costs associated with that, whether it's paying the spread or anything else. I only see the net overhead of the mutual fund as a whole. (And, as I noted, for a lot of 401k's, like mine, I don't even see that, because the 401k mutual funds are no-load.)
The ability of your mutual fund to provide low fee investments is directly related to their execution costs to trade to rebalance their portfolio.
Whether it is a direct line item on your prospectus or not is immaterial. No load mutual funds are more successful when they have lower execution costs. The biggest driver to low execution costs is an electronic market and then the bid/ask spread. HFT enable both of those low costs.
HFT doesn't enable electronic markets, it drives down the cost of them. The reason for this is that HFT participants are high volume users of electronic market infrastructure and have a vested interest in making that infrastructure cheap.
The speed for a market maker (largely passive and posting limit orders) isn't about how fast you can place a limit order, it is about how fast you can cancel it. It is a defensive mechanism against predators who are even quicker (arbitrageurs, aggressive HFTs).
Imagine that there is a "true price" of some security at price x, and your bid and your offer are positioned around that true price at x - d (your bid) and x + d (your offer). So your spread is 2d. Then some external (public) event happens, which moves the true price by an amount greater than d. Let's say it is a big event, and the true price falls by 4d. Now there is a race. If you don't manage to cancel your bid in time, a predator will 'pick you off', take out your bid, and they'll sell to you at (x - d) when the true price is now (x - 4d). The predator has made an immediate (paper) profit of 3d, and you've made a loss of the same amount. If you're fast enough, you can successfully cancel your bid and repost it further down the book at x - 5d.
Now, as a market maker, a liquidity provider, you could increase your safety margin by quoting a wider spread, setting your bid and offer at +- 2d instead of +- 1d. Now it takes a larger move before you're in danger of being "picked off", but you'll also attract less customers and make less profits. Speed (latency) is directly correlated to how tight a spread a market maker can quote.
What benefit do tighter spreads and faster response times provide to ordinary investors, like me with my retirement fund? I understand how they benefit the market makers; you've explained that. But all that really determines is who takes losses when an exogenous event that affects the underlying fundamentals of a stock gets reflected in its price. That's a zero-sum exchange: the market maker's loss is the predator's gain. How does all this create value on net?
So you've accepted that greater speeds allow market makers to quote tighter, so the answer is pretty simple. The tighter a market maker is quoting the less money he is making per transaction (smaller spread), so less money is leaving the system and going to middlemen. The speed allows them to undercut other market-makers and steal their 'flow' (customers) while still being able to avoid predators, and so they offer the service cheaper. The service being the provision of liquidity to bridge time gaps in a continuous time market.
If you assume that the number of 'real' trader (not middlemen) who want to buy or sell is constant (not necessarily true, but assume it for a moment), then it stands to reason that the tighter the spread, the less the market makers are profiting. At a small enough spread, they make no profit, since the profit from the flow on both sides is cancelled out by the losses from adverse selection (trading when they couldn't cancel in time - being victim to a predator). But if they can increase their speed, and minimize their adverse selection losses, then they can quote tighter and still make a profit. And do it with machines, and cut overheads of hiring humans, and you can quote even tighter since you need less trading profit to make a net profit. So this is the value generated, automation + speed = the smallest amount of frictional costs being extracted from the market.
> automation + speed = the smallest amount of frictional costs being extracted from the market.
Ok, this makes sense. But it still leaves the question, how much are the frictional costs decreased by HFT, and is that benefit enough to offset the social costs?
Such as? All I heard was smart boys and girls go and do a quantitative finance degree instead of Solve The World's Problems degree.
And that's bullshit. The social cost of _not_ providing other, better alternatives to our youth (downsizing NASA, underfunding research and scientific endeavors), lack of patent reform, lack of bandwidth and unified cross-country 3G/4G has much worse costs.
> smart boys and girls go and do a quantitative finance degree instead of Solve The World's Problems degree
No, they go and do a Solve The World's Problems degree and then end up in quantitative finance because they can't get rich solving the world's problems.
> The social cost of _not_ providing other, better alternatives to our youth (downsizing NASA, underfunding research and scientific endeavors), lack of patent reform, lack of bandwidth and unified cross-country 3G/4G has much worse costs
I think we're in violent agreement. This is exactly the social cost I was talking about. Our society is run by people who can't be bothered to compensate someone properly for curing cancer, but who will throw millions of dollars a year at someone for shaving a few microseconds off an HFT algorithm. That makes no sense to me.
Why are things this way? Lots of reasons, of course, but one of them is that, whenever anyone suggests that maybe it would be better if smart people were directed into more productive endeavors, the financial people have kittens and say it's going to wreck the world's economy if we don't shave those few microseconds off the HFT algorithm, because there won't be enough liquidity or something like that.
And that is bullshit. Sure, if we were at a point where all the big problems were solved, then yes, making tiny incremental improvements in the price of liquidity might be near the top of the priority list. But as things are, it should be somewhere around 56,732nd place. The problem to be solved is that there is huge, huge value to be captured by things like curing cancer, much, much more value than there is to be captured by shaving microseconds off HFT algorithms, but nobody knows how to capture it. How is making stock trading a fraction of a percent more efficient going to fix that?
If you need to buy/sell right now, having tighter spreads means that the price is cheaper for you. Your retirement fund, they need to buy/sell right now all the time, to re-balance their portfolio, whether for risk or for benchmark tracking, cash allocations, etc.
Saw your response and I'd offer the following:
A) the advantages that automation bring to the markets impact every single trade there is. Even if we take a very tiny sample of people who need to rebalance their EFTs today. Each day it is a tremendous savings.
B) HFT is a tiny part of the finance industry, yet it has had a huge impact on the cost of trading, to the end result of trading being phenomenally cheaper than it has ever been.
Is that enough of an advantage to outweigh the cost of the small number of intelligent people in HFT? I don't know, but it certainly seems easier to justify than the phenomenal amount of capital spent on internet ads. I'm a bit of a free market capitalist, so my biased response is how else should we allocate people's output?
I don't actually have a sense of how big or small the number is as a percentage of intelligent people. Particularly when you observe that, as you say, HFT is just a tiny part of the finance industry. There are many other parts of that industry that are also sucking up smart people to do things that, to me, have very little value compared to the other things those people could be doing.
> how else should we allocate people's output
Sorry to redirect again, but see my response to pas upthread. :-)
Again, looking at your other posts, I can only assume either A) you socially value internet ads much higher than most people who claim to value social equity over profit or B) you underestimate how many resource we as a society are throwing at internet ads or C) you over value how many resources we as a society are throwing at automated market making. In any case, Google by itself, whose entire revenue model seems to be based on internet ads, makes more profits in a year than the entire HFT industry does.
So it's hard to have the "it's terrible so many smart people are doing X" argument when our society has made it pretty obvious we value selling internet ads as high as any other possible commercial enterprise.
Me, I'll take the other position, which is that yes, the markets are bad at allocating capital, but they are better than our alternatives, and the silly little pocket change inefficiencies that go into speculative market making are a small price to pay for all the advantages of a modern liquid, price efficient market.
How are you getting that from what I said? You could s/HFT/internet ads/ in my post in response to pas and it would be pretty much the post I would have written about internet ads if that had been the subject under discussion. (Well, not really, because you'd have to change other things as well, but hopefully you see what I mean.)
> markets are bad at allocating capital, but they are better than our alternatives
This I agree with. But markets are as bad (or good) at allocating capital as the people who participate in them. In other words, markets reflect the values of the people who trade. I wasn't ranting about markets not being able to properly compensate curing cancer because markets are bad; I was ranting about people's values being so screwed up that the price signals they are sending into markets are making people rich for HFT and internet ads but not for curing cancer.
It's implied by what you didn't say. You don't require technologists involved in internet ads to justify their existence whenever a post about google comes up. Nor when news breaks of a new internet startup being acquired. I'm picking on you a little bit as a proxy for the entire HN community, as there seems to be a bias against HFT and towards internet ads that is a little unseemly.
Further, I'm not convinced you understand the scope of money involved in HFT vs cancer treatment for instance. When Roche bought genentech (a cancer research company) they paid 46 billion dollars for the privilege. Knight Capital Group, one of the biggest players in HFT is worth around 1.4 billion dollars. The market seems to be allocating resources correctly. The PR companies on the other hand...
> You don't require technologists involved in internet ads to justify their existence whenever a post about google comes up
Wow, you've gone through my entire HN corpus to verify that?
> I'm picking on you a little bit as a proxy for the entire HN community
I would prefer that you didn't do that, since I am not a good proxy for any community that thinks figuring out ways to get people to click on ads is a good use of smart people's time and energy. (Btw, I'm not sure that the entire HN community is such a community; a part of it may be.)
> The market seems to be allocating resources correctly
The numbers you give are interesting, but I'm not so much interested in money as in human resources. Genentech may have cost $46 billion, but are they paying smart people enough to get them to do cancer research instead of becoming quants?
Some of this may be a perception problem, as you imply by your comment on PR companies. What data I've seen on where smart people can get paid the most indicates that it's not cancer research. But I'm sure the data I've seen is incomplete; I would love to see more comprehensive data if there is any.
There is a (false) assumption among laypeople that limit orders "no longer work".
This (false) assumption is propagated regularly by articles and books wherein the same pattern repeats itself:
1. limit orders are defined
2. "... but on that morning Joe Trader was shocked to see that his limit orders weren't working!!"
3. description of Joe Trader NOT using limit orders ... or using them and canceling them and chasing prices ... or whatever.
I've seen this pattern in a lot of media since the Lewis book. In reality, limit orders haven't changed and you can set a limit and go to sleep / go on vacation, just like you always could.
I understand that currently the game is played with high liquidity. The alternative I'd like you to consider would consist of "real" investors with no need for split second decisions and liquidity. Give it a thought.
The 7-11 provides the trade functions "assortment of goods" and "transport" and as such is valuable. The analogy to HFT does not hold.
>The 7-11 provides the trade functions "assortment of goods" and "transport" and as such is valuable.
"Large supermarket chain" also provides the trade functions "assortment of goods" and "transport", so what exactly are you paying your extra 50c for? Liquidity.
What a bunch of assholes those 7-11 types are. Charging us an extra 50c for milk. The nerve.
See, as a true HNer (heh..), I would like nothing more than believing that a high tech software operation that makes money and is run by nerds is a force for good.
However, I'm afraid your proof-by-analogy does not work. If you actually know what you are talking about, why not explain the current financial system clearer and with more information?
In turn I promise not to burden this discussion with a dissection of your analogy, or even my own set of analogies which would beautifully fit my current bias, and which you would surely reject outright.
Another way to describe "liquidity" is inventory risk insurance. Large investors (a term that doesn't actually mean anything) have different time frames for buying and selling their product. If a large participant has to hold onto their portfolio longer than they want this will change the "natural" price they will be willing to accept.
Market makers generally, smooth these time disparities. As long as the smoothing is cheaper than the price disparity then it is a net positive for the "real" participants. How do we determine if this is cheaper? Quite simply, the market! You can trade non-liquid products electronically. The price spread his much higher in those products.
> Remove the liquidity (which comes at quite a steep price it seems),
The price of liquidity is less than the inefficiency (loss) that comes from an illiquid market (people who mutually want to execute a trade but cannot). This is rather straightforward to demonstrate mathematically.
Just because HFT extracts a premium (similar to a transaction fee) on the spread doesn't mean both parties (and the rest of the market) can't still benefit.
> The immense effort spent on HFT could now be concentrated on a) finding things worth investing and finding better ways to do so b) create more things worth investing in
This is analogous to the argument often levied by non-techies at the entire tech sector: "They could be developing software to cure cancer, so why are they creating more apps to send selfies to their friends?"
> This is analogous to the argument often levied by non-techies at the entire tech sector: "They could be developing software to cure cancer, so why are they creating more apps to send selfies to their friends?"
And IMO that's often a good argument, though it's not really an argument against the techies but against our society as a whole. The fact that techies can get rich making selfie apps much more easily than by curing cancer, is a sign that incentives in our society are screwed up, in much the same way as the fact that smart people can get rich by running HFT firms much more easily than by running companies that directly produce goods and services.
I fail to see how the "real" investors which I described above cannot execute a trade. Could your mathematical model describe some hypothetical situation?
The "premium" could then be split between the both parties. The cake is larger (but maybe not split as evenly).
The argument about webapps vs. curing cancer is also levied by techies right here on this site and might be valid, (yet weaker, since webapp-big data might eventually help with medical-big data?). Either way you have not made an argument for HFT with your last sentence.
Mr. Brown is a business owner and has his assets tied up in inventory that will produce future cash flows (maybe a farmer). Mr. Brown's inventory is in an incredibly volatile asset. He wants to reduce the volatility of his asset, so he hedges by some sort of asset that has a negative correlation with the value of the inventory he is selling. He is not an investor, he is a hedger. Now, this particular hedge product Mr. Brown chose does not have a high volume on the exchange in his local country, but in another country, it is much more liquid. Without hft, he is looking at significantly high prices to buy this asset because of the wide bid-ask spread (he will have to cross the spread to place a market order). A hft company can look at this as an arbitrage opportunity. A hft can take the market on both sides (become the highest bid and the lowest ask) because they know they can buy the asset in the foreign market, sell it in the domestic market, and then convert the currency back to their desired currency. They will obviously have to do the math, but there is most likely a risk-free trading opportunity for the hft. As more hfts come into this domestic market, the bid ask spread will become smaller due to competition. The people that lose are the current market makers.
Just a little nit-pick. The trade you are describing is not risk free. It has inventory risk, FX risk, counter party risk, operational risk, etc. It also has a cost of entry. All of those things still exist in the HFT space.
Again, back to my milk analogy further up, in theory you could get your milk direct from the farmer and cut out the middleman. I'm sure some of them go to markets and set up stalls and such. But the reality is, it isn't the farmer's game to do that. Every moment he holds that milk he is carrying inventory risk (it could spoil). And if a buyer doesn't turn up, then it definitely will spoil. Instead he offloads it to a middleman who is equipped to handle it (refrigerated tankers, etc.). Now in this real world example, the bulk of what the middleman is paid for is value-add (transport, pastuerization, bottling, etc.), but there is also the payment for liquidity. These companies (i.e. Fonterra) will be there to take your milk every morning. So some of what you pay for is that liquidity.
While I get the general idea of liquidity I really can't get my head around how it is supposed to work out in practice, especially in HFT. So let me make the question a bit more concrete.
A very illiquid market, I want to buy for $9, somebody else wants to sell for $10, nothing else. And now? How does HFT help? How does a traditional market maker help? He buys for $10 and sells for $9 and then somehow recovers the loss?
Market makers are not charities, they're there to provide a baseline for markets that don't have that many orders available (liquidity). This liquidity allows people to enter and exit positions more readily, encouraging real entrants to start trading the once quiet market.
People are more likely to trade if they feel a lower risk of being stuck in a 'bad' position.
Many times, exchanges actually pay market makers to provide liquidity to try and kickstart a particular market.
Liquidity directly relates to the number of participants (or quotes) in a market place. Let's think about a simple market, where the price of a good can go from 1 to 10 dollars. One person is looking to sell the good for $8, while another person is looking to buy at $2. The market currently has a bid-offer spread of $6. With only two participants in this market place, they will have to meet somewhere in the middle, or never trade. The seller will lose value because he has to sell at a lower price, while the buyer will lose money by paying more. However, if there were more participants in the market place, all with different ideas about how valuable the good is, the buyer is more likely to get their $2 price while the seller is more likely to get their $8 price. For example, if a new seller joins the market at a price of $6, the person looking to buy can now do so at a cheaper price.
Basically, with more participants, the bid-offer spread will begin to close, and trading will occur at more prices.
Liquidity refers to the fact that when you want to sell a stock, you can sell it immediately.
The more liquid something is, the closer it is to cash.
Think about your car -- if there was a public price for your car and you could decide at any moment whether you wanted to sell for that price, or close to that price, that would be a benefit to you. But instead selling your car is quite a chore and you are never sure whether you are getting ripped off or not. A liquid market has a public price and a degree of fairness and confidence baked into that price.
In order to provide liquidity market makers agree to buy stocks from anyone who will sell them, and then take care of finding the buyer on the other end. The difference between what they buy it for and what they sell it for is called the 'spread' and it is how market makers traditionally have made money. The spread is also the fair fee they charge for the service they provide, which is liquidity, and the risk they take. If market makers did not know for sure that someone else would be willing to buy the stock right away, that they might have to sit on it, they would be taking on a very large risk. The more people that buy and sell, the less real that risk becomes.
HFT makes their money, generally, by front running. Front running is the practice of buying a stock because you know the California Pension Fund is about to buy it, and so you know the price is about to rise. You can immediately turn around and sell it when it goes up.
Front running by a bank is illegal, but it is widely suspected that many banks engaged in the practice for years. It is illegal because the bank is supposed to be acting on their client's behalf and in their interest, so when they are trusted to place the order, they first buy some of the stock for themselves or for the bank's inside funds. It is basically insider trading.
When HFT engages in front running, they have detected that an order is coming down the pike, and they jump in front at the last nanosecond. The reason they spend so much money trying to do this is because it is essentially fool proof when done correctly. You can argue that by not taking any market risk, and not holding the stocks, they aren't providing liquidity, they are parasites on real stock traders and they are essentially using inside information. But its not that cut and dried because there is almost no way to eliminate the problem. At least HFT reduce the time window during which someone's signal to purchase something can be exploited.
It used to be that humans managed all of this. Large brokerage houses would be responsible for executing these trades, and it was common practice for brokers to tip each other off when they received large orders. These tips were very valuable because they guaranteed a winning trade. If you ever wondered how meat-head frat boys could make millions of dollars on Wall Street despite not being especially intelligent, this was a big part of it. At least now the people who are scalping everyone's orders have to be brilliant.
Right. Instead they have small orders placed. As soon as that order is fulfilled on that exchange, depending on algorithms, the HFT will look for opposite orders on other exchanges, and buy, very quickly. If it can do so before the matching engine gets there, then the price can be driven up.
For example, you want to buy 10,000 shares of X.
Matching engine finds 100 at 9.98 - it’s the NBBO (best buy offer), it’s required to execute that, and carry on looking for ways to fulfill your order below or at your bid.
Meanwhile, the HFT offering those 100 shares sees this, decides its a parcel of a large buy, and if it has a quicker route / algo than the matching engine to other exchanges, goes there, executes immediate buys at say, 9.99, then immediately raises their price to make a profit on the spread.
Except that's not how the latency arbitrage trade actually works.
What is actually happening is that the HFT is offering shares on all the exchanges on both sides of the bid/ask spread at the same time. When they get filled on an order at a price on 1 exchange it is used as one of many signals to indicate if their pricing is accurate. If their algorithm thinks that fill indicates a price movement (for instance if the entire level was removed) then they will update their prices on all the other exchanges accordingly. If the algorithm doesn't believe this indicates a long term price movement they won't (or they may even re-up at the same price on the exchange they got filled on).
No one is going out and buying ahead of someone else because that is too expensive (you have to pay the bid/ask spread to do it) and too risky (you might be wrong about your price determination).
Finally, just a detail, you can typically opt out of NBBO matching if you want.
I have a hard time to imagine how that works out. After all they are not interested in buying any shares so they have to quickly sell everything they buy effectively canceling the buy. How does that provide liquidity?
I guess so. The problem is I really can't get my head around how market makers are supposed to work. If my stock performs well it should be easy to sell it for a fair price in case I need the money because somebody else will happily take the opportunity to make some money. If the stock does not perform well I will feel more comfortable with a market maker offering to buy the stock but what is the incentive of the market maker to buy my stocks? Almost the same goes for the other way - why should a market maker sell a well performing stock to me? If on the other hand the stock does not perform well but I really want some I should be able to get some anyway because others will be happy to get rid of them. I am always tempted to call bullshit on stock market liquidity but then the term is all over the place and I am pretty sure that not everybody besides me is an idiot and so I guess I am just missing an important point.
Market makers are licensed by and incentivized by the exchange. They will usually have cheaper or no transaction costs, be given order priority over other market participants, have more relaxed rules on quote display times, or any other combination of advantages over all other market participants. In return they are required to take both sides of the market (eg: buy when everyone else is selling) and respond to a minimum percentage of quotes (eg: trade when no one else wants to). For the exchange the advantage is that they provide a minimum level of liquidity at all times which is what attracts traders to the exchange.
They will usually have cheaper or no transaction costs, or, in some cases, a negative transaction cost--paid by the share traded by the exchange to ensure liquidity.
I wrote this a few months ago, trying to explain market makers; I lightly edited it to make it make sense on this thread.
At any given point in time, the "true" (buy-side) investors are likely to be relatively far apart on prices. This is especially true in thinly-traded markets. For example: consider housing, which is the canonical example of an illiquid market.
I know the value of my house (it's around $300k). Say I want to sell it. The more liquidity I need, the worse price I'm going to get. Anyone with any intuition for the housing market knows that if I have to sell my house tomorrow, I am going to get a god-awful price for it; the "tomorrow" price for my house is many tens of thousands of dollars off its true value.
This creates huge problems for homeowners. The obvious standard advice for sellers is to wait patiently for the best price, counting on many weeks or months before a reasonable offer arrives. But I have to pay money every month I hold on to the house. If I need to move immediately, for instance for a job, or because my financial circumstances have changed, I might need the house to sell quickly, and because houses are illiquid I have to accept a crappy price. Or consider trends in the market: by forcing me to hold the house for months rather than days, I'm maximally exposed to swings in the market. So if Chicago housing prices crater while I'm trying to sell, the extra time it takes to unload the house takes that price swing out of my hide.
(You can easily see how the same thing happens in reverse in "hot" markets like Palo Alto, with the buyer now assuming the role of the hapless Chicago seller).
The exact same thing happens in the public markets; it's just not as intuitively obvious because we're working in smaller deltas of time and price.
But the public markets have a huge advantage that the real estate market doesn't: market makers.
Imagine if houses traded (were bought and sold) so frequently that a smart company could make good educated guesses about the current value of any given house. Imagine if that company would on any given day snap up a house offered for sale. That company wouldn't pay my asking price for my Chicago house, but it would pay something much closer to it than the "true" market would. After buying my house, the company would immediately offer my house for sale at any price greater than what it bought the house for. I wouldn't care, though: either I'd be thrilled for the opportunity to sell my house quickly and painlessly, or I just wouldn't accept their offer and instead do what I do now, which is to wait for the best offer.
Assuming the housing market makers were smart, the money they'd get by forcing me to accept a lower bid would be "free money" they get simply by being smart about valuing houses and having capital available to deploy. We'd all be a little irritated at them for scalping distressed buyers and sellers. But something else will inevitably happen: other smart firms will smell the free money, and they will compete for it. To capture a share of the premiums, all they have to do is offer a slightly more favorable price, so that's what they'll do. Over time, the money the housing market makers will get less and less "free", and the price penalty for immediate liquidity will get lower and lower.
In my fantasy real estate market, I now have the opportunity to avail myself of a reasonable "market" order, or to buy or sell on a "limit" price. I could do that if there were housing market makers, but in their absence I can't, because the price hit for selling tomorrow is too great; it might be a 25% discount on my asking price, or even something close to 50%. That's the cost of illiquidity, or, stated more directly, the value of liquidity.
It's probably also worth saying that liquidity is a presumption of the public electronic markets. Because it's taken for granted, there are whole trading strategies (hedging, for instance) that depend on its presence. These trading strategies face execution risk: if they can't bail out of a position within a specific window of time, they incur losses for the trader.
Thanks, that helped. My major mistake was to assume bid and ask prices will not change and ignored that you might really need the money and are willing to reduce the asked price. So in essence market makers a paid to hold stocks they don't really want and take the risk of unfavorable price movements.
One explanation I've heard for HFT's profitability (which I'm too lazy to source at the moment) is that it's also an artifact of the granularity of prices permitted by the exchange, just like the 1/8 limit.
Instead of competing to save you a few more slivers of a cent per share, they compete to fill your order a few microseconds faster (at the next-highest price permitted). Seems most traders would prefer the money.
I always see this argument. Market makers provide liquidity, and yes, HFT often act as market makers. However, I am not at all convinced that the high frequency element adds anything useful to the equation. I don't care whether my trade executes in a microsecond or a second. In fact, I often leave limit orders open for DAYS in order to get the price I want. I am also not convinced that most HFT are truly acting as market makers.
I think the most charitable thing that can be said about HFT is that it narrows the spread. In order to make it worth their while, the HF traders then have to make a lot more trades. Whether this is adding any real value to the system is harder to say because while the spreads are smaller, the volatility is higher.
It doesn't make much sense to challenge the liquidity provided by automated market-makers while conceding that they reduce spreads; spreads are essentially a proxy measurement of liquidity.
I didn't say that they didn't provide liquidity; I said that I am not sure that the additional liquidity they provide actually matters to anyone other than high speed traders.
> The purpose of the "speed" is to elicit information.
Not quite. HFT uses information that you probably don't have, and acts on that information using software and hardware that you probably don't have[0].
You can obtain that information and that software/hardware - it's just expensive (high barrier to entry)[1].
As explained above, HFT exists because portions of the market are not perfectly competitive in the economic sense[2], but that's always been the case (and arguably was more so before the advent of computerized HFT).
[0] It's not so much that you don't have the information - it's that the latency is too high for most players.
[1] At least some portion of these costs are very real - the cost of maintaining low-latency connections between exchanges, colocating hardware, etc.
The HFT traders are most certainly getting information from the speed. They are submitting flash orders of a low quantity to gauge the demand of a stock, the if a buy order is detected, they buy all of the stock from the other exchanges.
Where does this assertion come from? If you take a look at the market book, then you might see that yes, most market-makers (liquidity) in the equity markets are high-frequency orders.
Language is such a tricky thing. 'Speed' is the time it takes to act on information, hence the 'high frequency' in the name to distinguish it from 'regular frequency' but it isn't the purpose. The fact that these traders exist, and their only motivation is to buy or sell at a marginal gain, means that there are always buyers and sellers for a given commodity in the presence of an HFT firm, they do provide the 'other side' of the buy/sell transaction and thus by definition provide liquidity to the assets being traded.
I read a copy of the Mentat prayer that had been written for traders somewhere but have lost track of it, that was something like 'by liquidity I give my transactions life, by speed I give them value, by value I make me wealthy' or something along those lines. It was much more clever than I could come up with, but modest google fu has not revealed its location.
The do not 'provide liquidity' -- the liquidity is already there. Suppose there is a stock selling at $100. You want to buy 100,000 shares, and there are people willing to sell. A HFT leaves some bait 100 share blocks for sale at that price. You buy those first, they use the advantage of speed to buy any other $100 shares available. They then sell you what you wanted to buy at $100.10.
What value did they add to the market? Bringing buyers and sellers together? Nope. Provide capital to the market? (Which is why stocks supposedly exist...) Nope. They end every day owning nothing.
Lewis's book also shows that many banks and brokers instead route some trades through their own internal 'markets' first, trying to find a match, instead of just trying to get the fairest price available.
Everyone gets screwed. There was a HFT trader who boasted that they went years without ever having a losing day. The only way to do that is in a market that isn't fair, where not all parties have equal access to information.
I can't reply to that thread (maybe it's because it was too long ago), but you draw a difference between someone moving a large block (as described in yummyfajitas' post), and someone trading a small size (50 shares). So let's say that someone does want to trade 50 shares, in your example, and breaks up their order into 2 25-share orders. At the point at which a 25-share order is executed on the market, the sheer size of the trade is not very much. It's not likely to signify a market move. At that time, nobody has knowledge that there is a second 25-share order heading to the second market. It is possible that an HFT player sees the first execution and takes out the shares at the second market, but with such a small execution, they are pretty much guaranteed to lose money.
No, that's not the distinction I was drawing. The distinction I was drawing was between the following two scenarios:
(1) Someone wants to move a block of 50 shares, but there are only matching orders in the order book for 25 of those 50 shares. The seller splits his order so that the 25 shares move; then either the other 25 sit there until a matching order appears, or the seller changes his offer price for the other 25 shares so that there's a match.
An HFT in this scenario is no different from any other investor deciding whether or not to match an order in the order book; he just moves faster. And since in this scenario there are no other matching orders in the order book without the HFT, the HFT adds liquidity in this scenario.
(2) Someone wants to move a block of 50 shares. There is no matching order in the order computer in which they place their sell order, but there are matching orders in other computers. The HFT sees those other matching orders before other market players, and matches them. Then he places his own order in the computer where the original 50-share sell order is, and matches it so it executes.
In this scenario, the HFT adds no liquidity; all he does is shift the profit on the trade from other market players to himself, by taking advantage of the latency between different computers on which orders are placed. The trade would have happened anyway once the two computers reconciled their orders.
The reason is precisely because HFT market makers are fast and sophisticated enough that they want to trade with all small block buyers, so they are resting orders at every trade-able price on every exchange they can.
The speed arbitrage comes from the fact that if they see a level being removed at one exchange it is a demand signal, so they change their own prices on the other exchanges. This is faster and cheaper than going out and sweeping another level (which requires paying the spread).
Essentially, there is no liquidity difference in this case, as all the same liquidity providers are there. They are just pricing their liquidity more accurately.
That seems to be a disputed issue right now; claiming that scenario 2 does happen in practice was, as I understand it, one of the main points of Michael Lewis' latest book on HFT. He may be wrong, but I don't think it's as simple as saying "they're so fast that they don't need to".
An HFT may have orders at every tradeable price at one instant, but what about the next instant when the set of tradeable prices changes? On an exchange with multiple computers executing trades, that is precisely the time when scenario 2 can come into play, and the fact that HFTs are faster than everyone else is precisely what gives them the ability to play scenario 2 in this situation.
> if they see a level being removed at one exchange it is a demand signal, so they change their own prices on the other exchanges
To be clear, my scenario 2 was not talking about matching prices on different exchanges. It was talking about a single exchange that has multiple computers executing trades. Conceptually, the single exchange is supposed to have a single order book, but because there is unavoidably latency between the multiple computers, computer A's current view of the order book may not match computer B's. An HFT that can detect the mismatch faster than the exchange's own computers can, can run scenario 2 in that situation.
"That seems to be a disputed issue right now; claiming that scenario 2 does happen in practice was, as I understand it, one of the main points of Michael Lewis' latest book on HFT."
Let me come right out and say it. Michael Lewis is not "wrong" because he never actually says that scenario 2 happens. The reason he doesn't say that, is that there is no proof that it happens. The reason he couldn't find proof is that it doesn't happen. Any cursory examination of the HFT side of the industry will show that it doesn't happen. He never even does this cursory examination. Go through the book, catalog when he talks to the HFT side. He doesn't do it. The closest he gets is interviewing a developer who is being persecuted by his own buy side firm. Michael Lewis implies that something happens but never actually shows it happening. He does this either due to incompetence, malice, or prejudice.
To prove my point, let's do a thought experiment. Let's say I have 2 HFT systems, each of which are equally fast and are equally good at determining if fills indicate true price movements or blips in the market, and are equally good at evaluating outstanding risk. HFT A's strategy is to quote "small" size in a single market and then when it sees fills it thinks indicate price changes to buy a level in order to raise the price in a particular market. HFT B's strategy is to quote at every level in every market and when it see's fill's that indicate price movements cancel it's order's that are at a bad price.
If their price discovery or risk algorithm are perfect the best case scenario is that HFT A and HFT B are paying the exact same price for the right to trade. If there is any slippage in their price discovery or risk algorithms, HFT B wins as it is not paying for the privilege to trade. QED, algos that don't pay up to jump priority queues do better than algos that do. In the blood thirsty world of HFT market making this inefficiently quickly leads to HFT A going out of business.
As far as your scenario about an exchange that is open to distributed systems attacks, that is akin to asking about a casino that has a secret flaw in their poker shuffling. Yes, that would give an unfair advantage to anyone that found the flaw, but it would also kill their ability to host poker games if it became public knowledge. IE exchanges have a huge incentive to make sure this doesn't happen.
> algos that don't pay up to jump priority queues do better than algos that do
It looks to me like all of this is addressed at the version of scenario 2 that I was not talking about, the one where the HFT is working across multiple exchanges.
> exchanges have a huge incentive to make sure this doesn't happen
I'm not sure the casino analogy quite holds here, but I'll have to defer any further comment until I can find the article I mentioned somewhere in this subthread, that went into how this scenario would work in more detail.
In response to situation (2), I want to first tighten the language - as it stands, I'm not sure if it's even possible. By computers, I think you mean exchanges (for example, Nasdaq, BATS, etc.) So somebody wants to trade 50 shares there. Let's say they want to buy at $10.00, and there's no sell orders on NASDAQ but 50 shares on the offer at BATS.
1) If you were doing this through a broker, and that was the state of the book, then they would just route your order to BATS (if I'm not mistaken that's a legal requirement for brokers).
2) If you instead placed the order to be on Nasdaq-only, that would lock (have buy and sell orders at the same price) the national book, something which is not allowed due to Reg NMS. So Nasdaq could not display that you want to buy at $10.00 anyway, and (I'm not sure how they implement their stuff) either hide your order or display it at another price.
So I'm not sure how exactly you're framing this situation. If the HFT's computer can see that your order exists, it's because it posted. If it posted, it could not have had the limit price that would've locked the book.
I may be misunderstanding what you mean when you say at the end that the "two computers reconciled their orders" - because nothing like that does happen. If they are computers at the same exchange, then no, that's simply not how matching engines work. If at different exchanges, then there is no reconciling, your order would not have posted at the right price, or your broker would have not tried to get you best execution on your order(which they are bound to doing).
No; as I responded to another post in this subthread, I mean multiple computers belonging to a single exchange, such as NASDAQ. In one of the previous HN threads on this topic, somebody linked to a post that showed the situation I'm talking about; I'll see if I can find it.
I started reading the Cringley book on IBM. It's a captivating and maddening read. By this story IBM management, trying to satisfy the desires of shareholders looking for earnings growth, is destroying the company.
HBS is graduating lots of portfolio managers and analysts who contribute to this phenomenon. Not so many HBS grads in HFT, based on the folks I know in the industry.
HFTs certainly aren't driving major global corporations into the ground by pushing for stupid forms of management like share buybacks, "outsourcing" core competencies, and the like.
I give this guy credit for referencing these impacts. I don't think HFT is really part of that trend at all. HFT is actually pretty benign when you dig into the details. Sure, they scalp pennies here and there, and it adds up. But the competitive pressures on technology force efficiencies to a point, which is good for the market.
I agree . I am far from an expert but I did not understand the connection between HFT and companies making bad long term decisions for short term. He said himeself that HFT firms don't care about the actual companies anyway.
The author seems to expect the reader to assume that HFT does not create value. There are a few points to back up the belief but he does appear to state it more as a reminder than an argument. He then goes on to pose what I see to be the point of the piece. A question about whether the short term thinking created by shareholder pressure may be creating the same situation.
Maybe IBM will never go out of business even after creating little or no value. But instead it will find methods outside of creating value in order to create wealth. Much like high frequency trading.
> it allowed the high frequency traders to peek at the ballots others were sending in to the newspaper before they arrived, in turn giving them the ability to cast their votes using information not yet available to the rest of the market.
From what I gathered from Lewis's book, the crux of the problem is that people were doing their trades in multiple exchanges and that HFTs were simply noting trades in the fastest exchanges to reach (like BATS) and using that information in exchanges that were farther away.
To take his analogy, it would be like using the results published in another newspaper the day before to determine your ballot choice.
There is a very deep misunderstanding of the purpose of stock trading. Somehow almost everybody assumes that the very purpose of stock investment is to time the market. But no, timing the market is only a mechanism of the market. Those who successfully time the market are more a part of the market mechanism rather than the customers/end users of the market.
The only real purpose of financial markets is risk management. Any human society (and non-human societies also) has to risk some capital to gain more capital/value. The purpose of stocks and financial markets in general is to diversify the risk, while still funding oportunities and it's the only proven mechanism by which a large portion of the population can participate in this wealth creation.
The value of risk management is hard to fathom. But if you are a programmer, you might want to try and simulate "gambler's ruin" and see what happens with high risk or low risk strategies. In this simple game, computing the right risk management is the difference between stagnation and exponential growth. In a complex economy, this seems also to be true.
Summary: investors who focus too much on short-term results are bad for the long-term success of the companies they buy and sell.
High-frequency traders are somewhat similar, in that they don't care enough about the long-term health of the companies they buy and sell. Therefore they are bad.
I'll agree with the first premise. But I don't think the 2nd bit follows.
The reason speculation based on short-term results is problematic is that it creates perverse incentive structures that discourage a company's management from thinking too much about the long-term success of the company. But that's only possible because speculators are able to create market demands that punish and reward specific kinds of behavior. But HFT doesn't really create demand; it only responds to it. So the analogy fits in every spot except the only one that matters for the sake of the argument.
Treating HFT as a proxy for 'Finance' is a flawed premise.
Without 'Finance', you would not have:
- Cheap mortgage rates - mainly due to securitization.
- Cheap loan rates - yes, car leasers, I'm looking at you....
- Rational commodity markets. Think Starbucks, every car/computer made today.
- You get the idea.
Certainly, some aspects of modern markets appear to add little value, but the bulk of 'finance' benefits not only companies but everyday consumers immensely.
-
Without market makers and other speculative market participants you would not have rational commodity markets or liquid price efficient equities.
HFT versions of those market participants are dramatically more efficient and fair than the participants they are replacing. That efficiency is shared with every other participant.
If you believe in public commodities and equities markets, it is extremely hard to argue against HFT and in favor of pit traders...
The 30-year mortgage is mostly a product of government. And in supply-constrained markets (most of the American coasts), low mortgage rates have mostly resulted in sharply higher-priced houses/land, which isn't good for anyone wanting to actually own a house. And that's one of the more benign results of mortgage securitization.
I agree about rational commodity markets. However, I am not sure everyday consumers are benefiting from cheap loans. I think cheap mortgages have driven up house prices, cheap student loans have driven up tuition.
I used to work at Merrill Lynch, I saw what the focus on short term results did and the culture it created. It was almost always about the quarterly results. I once saw an entire team of bond traders fired because their quarterly results weren't up to scratch, only to literally be hired back within days at a much, much higher cost, because the market suddenly turned almost overnight and Merrill Lynch was caught without a bond team in that area and had to frantically scramble to get one at almost any price.
That particular team had been highly profitable, but their last quarterly result were poor. It mattered not that overall they were still vastly ahead in terms of profit for the company. They had a bad quarter and the next was looking poor, so they had to go, in order for the managers to have something to say to their investors.
If the share price dropped a few cents or if the results weren't up to or exceeding market expectations, they would fire a few thousand people. I kid you not, this is how investment bank culture works. You are only as good as your last quarter.
Traders would take very risky positions with huge or unknown long term risks, in order to make short term gains - for which they are richly rewarded. No one cared that this wonderful basket of exotic options was actually full of potential toxic waste, as long as it was printing money right now. By the time it exploded, most of those traders and managers would have made their millions and be retired or working somewhere else.
It was and probably still is this intense focus on the short term that caused ML to very nearly go under and why ML is now called Bank of America ML. I doubt they have learned and the next black swan event is going to take them out, or the one after that.
> Now, there are some rockstar CEOs — who oftentimes happen to be founders, such as Bezos, Steve Jobs, Reid Hastings — who have the ability to resist the pressure that the markets put on them.
Another possibility the author should consider: there are some CEOs who have the ability to convince human investors that their long-term visions are worthy and deserving of steadfast support.
What the author fails to address is the fact that not every chief executive has a great long-term vision, and not every chief executive is capable of persuading investors that he or she is capable of realizing such a vision. The implied notion that investors are somehow obligated to give the companies they invest in an arbitrarily long period of time in which to execute regardless of evidence of tangible progress is foolish.
The Oslo Børs stock exchange in Norway has a fee that will affect unnecessarily high order activity in the stock market, to prevent high freqiency trading. Here's more info:
Time and time again this discussion is hampered by the term "high frequency trading" which is much too general. An instance of a HFT algorithm behaving poorly should not condemn the practice as a whole.
Saying "HFT is bad, just look at electronic front-running" is like saying "Sorting algorithms are slow, just look at bubble sort".
So the problem with traditional buy and hold investing based on fundamentals is that it's very slow. I don't mean slow in terms of latency and distance to servers, I mean that financial returns accumulate slowly. Investing in the market as a whole will return roughly 10%-15% depending on the year. This beats what most individual day traders and money managers can do, but is nothing close to what a well positioned HFT trading firm can make.
"Sophisticated" trading strategies exist because the finance industry has focused itself on one goal to the exclusion of all others: Make as much money as possible. It doesn't matter if it's a zero sum game or if the most fit companies are being selected based on research in a darwinian process. A rational actor is going to chose 30% over 10% every time.
The bottom line is that predicting the success of a given company is hard. Predicting what the other actors in the market are going to do is also difficult, but is much easier by comparison. Asa participant in the market you want to be in Keynes' newspaper contest, not in the business of making broad predictions about the future. As a member of society we'd prefer that capital be allocated to the firms that will use it to our most benefit. I don't think anyone wants to use the law to change the current system so that it is more useful to society. A lot of people are also unhappy with a lot of finance being in its own bubble of "irrelevance". This is a decision that we're all going to have to make about how we want the financial markets to work. Things seem to be working out ok now, in a general sense, so change might be a long time coming.
>This beats what most individual day traders and money managers can do, but is nothing close to what a well positioned HFT trading firm can make.
I don't know if this is actually true if you average up all the successes and failures. Probably more like, for every successful [daytrader|money manager|HFT firm] there are 9 that lose money or go bankrupt and nobody ever hears about again or includes in average return calculations.
It's just that the rewards to the few successes are so tantalizing, there's never any shortage of folks jumping into the game trying their hand at it.
>The bottom line is that predicting the success of a given company is hard. Predicting what the other actors in the market are going to do is also difficult, but is much easier by comparison.
Not sure I'd agree. With companies you can at least throw some old school research and hard work at the problem. Difficult but at least nobody is out to eat your lunch there. Its just difficult as opposed to cut-throat.
>rational actor
Can you really count on that if 60% of the market is AI and a sizable chunk of the rest is trading on "superior"/inside info?
I really can't take any lessons derived from Lewis's book on HFT to heart, because every HFT friend I talk to (even the ones who are quite objective about its role in the markets) insist that much of the book is simply wrong.
The major problems with the book fall into 3 major categories:
A) He implies (but never proves) that HFT market makers use low latency connections, to buy shares ahead of other market participants, to sell back to them risk free. This is not how HFT market making works. HFT market makers are pricing on all the exchanges at the same time. So what they are doing is not buying something and selling it back to you risk free, but changing the price of their own offering to reflect new demand.
B) That any of this is secret or requires insider information. It is all available on public websites, including governmental agency ones (for instance all "exotic" order types go through a public approval process). Further, in the book the only proof of ill gotten proprietary information being used for profit was from the supposed "heroes" of the story.
C) That HFT is a large force that abuses it's power to take advantage of buy side participants who have "Main streets" best interest at heart. In fact, HFT firms are the small guys bringing efficiency at the cost of the powerful entrenched buy side middle men.
I see HFT as a form of arbitrage. If you have huge volumes you can make money on FX markets too - but basically you get price convergence. The big players make enough of it to make it worthwhile, but it's really on the fringes.
I have never heard of any firm making consistent positive returns on foreign exchange markets, in fact, it is often cited as an example of an efficient market, because it is so unpredictable. Alan Greenspan's autobiography mentions a discussion he had while working at Goldman Sachs, where he asked them how GS was making a consistent profit on FX (because no one else seemed to be able to), and they responded that it was because they got a commission from making the trades on behalf of the customers, but GS never traded FX on its own accounts.
The first part about Keynes is depressing. It presupposes there is only one winner. This is what anti-capitalists want everyone to believe. One winner, everyone else is a loser.
HFT is completely distinct from front-running.[1][2] Front-running means that you (as an investment bank or other trading company) buy and sell stocks according to the orders you have received, but not yet executed from your clients (usually large institutional investors). HFT just means that you are quickly executing trades based on some algorithm(s).
Three words on this "race to irrelevance": lead versus leave.
Our society is in a state of secessionism. Much of the escalating economic inequality comes from that impulse. The rich of yesterday (1945-73) saw themselves as leaders of the society. The rich of the new Gilded Age (1974-2014+) have given up and just want to escape it. They want private schools, country clubs, and closed social networks. They don't want to lead the masses, they want to leave them.
Silicon Valley has been making its secessionist impulse visible of late, but HFT and "high finance" show a different secessionist tendency: the desire to get outside of any given industry or company and "float among" them as a financier. The brightest young people are being told not to join the regular economy with the proles, but to become part of an elite system of hedge funds, private equity shops, and overcapitalized "startups" whose products are meaningless other than advertising expenses to get middle-management positions, 10 years earlier than otherwise, for the founders.
The smartest people have given up, sadly, on leading. This makes "leave" the attractive option. If you become a management consultant or investment banker, you don't have to commit to one industry or business. You float around until you make friends who can place you at high levels.
After spending 8 years in a number of places but much of it in "the real economy" I can't say that I blame the leavers. The leavers are now ahead of me, career wise, as venture capitalists and the like; and there isn't much out here in "the rest of the economy" to lead. Perhaps paradoxically, the world ends up being run by leavers, because average people don't want to be led by the highest level of talent; they want leaders they can relate to.
1. HFT does NOT take anything from most investors most of the time. Most investors trade on a much longer investment horizon than any HFT firm. If you are a normal investor, you can lump up most of HFTs into the same bucket as the exchange itself. Most of them engage in some combinations of (i) pure arbitrage (ii) very short-termed statistical arbitrage (iii) market-making based on (i) and (ii). None of this is really that relevant to most investors. The only time HFTs can screw over a lot of investors is when their software goes awry. But this is a risk that any computerized system has, high frequency or not. If there is a major bug in Chicago Mercantile Exchange's matching engines, that would be a total disaster.
2. HFT firms do have huge execution risks: if you are making a two-sided market, there is a chance you can get "swept", meaning when the value of the underlying moves faster than you can react, you get filled on one side of the order without a realistic chance of hedging for a profit. HFT firms do a ton of research into estimating their execution risks. (edit: This point is often glossed over, making HFTs look like this evil superpower group of nerds exploiting other investors. That's not the case)
3. Finally, most HFT traders don't go into the profession solely for the money, just like most people do not apply for YC solely for their passion. Most of my former coworkers weren't that greedy and led pretty modest lives despite making hundreds of thousands of dollars. For them, high frequency trading had a locally optimal balance of tackling intellectually challenging problems while getting paid handsomely.
I won't say HFT is the most valuable thing that its practitioners can be doing. I think the (trading) world would operate just fine without all these micro-second level transactions. But they are NOT the next subprime mortgage, and I just hope folks stop commenting on stuff they have no clue about.