What people don't seem to get about HFTs is that retail investors cannot and should not attempt to compete with them or trade actively at all - HFT or prop trading is a big no-no for the unsophisticated.
Historically buying into an index fund during a crisis is a perfectly reasonable way to achieve relatively easy alpha above the mutual fund/bond rates (~12% per year).
This type of investing (aka diversified value) is completely separate from HFT. As far as many retail investors should be concerned HFTs can duke it all out between themselves as much as they want - it really doesn't matter and is irrelevant to your long term investment decision.
Indeed HFTs provide the benefit that during times of crisis they provide plenty of buy side liquidity for you to get in at a lower spread and a lower spot price relatively easily.
Are you saying that HFTs have no impact on Alice and Bob regardless of how prudent they are about investing? In addition to ETFs Alice and Bob also have market exposure through a company 401k. Are the 401k managers and ETF managers immune to the effects of the HFTs?
On the page[1] first documenting this algo NANEX says the following:
"We believe that this algo will continue to grow and if left unchecked, could very well contribute to the next flash crash because it removes precious network capacity and provides zero economic value such as price discovery."
It has no effect on Alice and Bob if they don't sell. It is favourable to them if they buy.
The stock price is just the opinion of 0.2% of people at any one time - and hence means essentially nothing to an investor. Does the spot price opinion/first impression of a person matter - or does the long term attributes of their character matter? Just because there are more opinions - being given at a faster rate - doesn't mean that a) they're right or b) they should be acted upon as fact and c) that they shouldn't be exploited for those of a more stable nature.
For example: I bought a huge amount of TSLA stock when it fell 12% in one day a week or so ago for no particular reason. I subsequently realised a 6% gain. I'm happy - thank you HFTs and short term traders - your vol makes my alpha.
No, he's arguing that events like flash crashes don't destroy wealth at all. If the market returns to its original equilibrium after a week or so, then its as if the crash never happened at all.
Just keep in mind, in the stock market, you haven't made or lost money until you close out your position.
There's as much wealth created when the price rises again as there is destroyed when the price first erroneously falls, all other things equal. What's really being described is turning a profit on prices that are, for whatever reason, set too low and return to previous levels soon after.
Indeed. No one should be pitied for losing money in the stock market. If you know what you're doing, you'll know that there's risk involved; if you don't know what you're doing, you shouldn't be picking stocks in the first place.
People who lose out by being on the selling-too-low side of arbitrage have nothing to complain about. If you had a stop-loss order, you cede your position to the possibility of being sold too low. Moral being that like you said, if you're investing in something that might warrant a stop-loss order, you should be sophisticated enough to use something better instead.
>> If you're investing for long term - you shouldn't be investing in things that require stop losses.
Except that in the long term good companies sometimes go bad quite suddenly. Frequently, the harbinger (e.g. CFO suddenly quits) will erase a lot of wealth quite rapidly. Stops are a good way to not have your portfolio blow up while not also having to obsessively follow the news.
As an individual investor, you're not going to beat the automated algorithms on breaking news anyway. Trading individual stocks is a fool's game. Buy and hold index funds.
Just to be clear, I wouldn't counsel racing algorithms (this should be obvious). But you want to be able to exit somewhat rapidly in case of catastrophe at a portfolio company. A stop can turn a big loss into a smaller loss.
>> Buy and hold index funds.
To each his own, but this strategy has been pretty easy to beat over the last 20 years (even easier over the last 10), for those willing to study companies at all.
>Indeed HFTs provide the benefit that during times of crisis they provide plenty of sell side liquidity...
Any proof to that claim? Liquidity evaporated during the flash crash. The SLPs have no obligation to provide liquidity and can simply pull the plug when the market is in crisis. [1][2]
Buy side liquidity means quick access to sellers at little cost and on favourable terms to you. This was always the case - HFTs just make it faster and easier.
So if you are a buyer in a crisis - you get a better deal with HFTs quickly matching supply/demand of large stock orders. If you are a seller - you're screwed either way - HFT or not (see history).
Buying in the dips might give you a median positive return (95% of the time you'll come out ahead) - but your expected return can easily still be negative due to what happens the other 5% of the time.
If you're following a very simple strategy based on past returns that seems to make money all the time, you should wonder what the rest of the market is afraid of that you can't see.
If you bought in at 200 after the market had plummeted from 380 down to 200, you'd probably be thinking your strategy is working pretty well - especially once it rallied back to 290 or so. But after that point you'd be waitinga LONG TIME to get your money back. It wasn't until 20 years later - the 1950's!! - that the DJIA finally sustained a level above 240 (and not before falling to 40 - good luck staying solvent through that!!).
And this is only because the US economy did eventually recover (thanks to World War II). Argentina's stock market never did recover. There's no such thing as "time diversification". In the long run, the variance of your annualized return increases: http://www.norstad.org/finance/risk-and-time.html
This is why the DJIA fell to 6500 in Oct 2009. If you bought then, you are probably feeling pretty smug now - but it's simply that the rest of the market was afraid of Great Depression II and you may not have even realised that it was a possibility.
To say that you can obtain a positive expected return by following any strategy that is solely based on what the price has done recently is just as "naive" as a retail investor who thinks they can beat HFT algorithms.
As the SEC says: "Past returns are never an indicator of future performance"
1: I've been long TSLA since the IPO - I simply bought more stock with cash on hand. I'm not just median riding - although that is a relatively effective strategy for high earning companies/growth.
2: Risk is risk - TANSTAAFL. Great Depression risk is there just as there is nuclear war risk. I take it because I can. I try and make sure I pay the right rates though.
3: DJIA is not the entire market - it's a highly constrained subset.
4: Following past strategies does have positive value - it's what investing (and everything thing you know) is all about.
You live and die by induction.
Thinking that you are high on your black swan horse by stating otherwise is pointless.
Decisions need to be made and money needs to be correctly invested under uncertainty. Taleb guys bore me.
During a liquidity crisis - aka any crisis - buyers don't get front run - sellers do. There is a reason for this - front running only occurs when you want something (liquidity) without having to pay for it first.
For example: You want to off load a metric ton of stock in a company - you tell your broker - he puts out a VWAP sell call to an algorithm - HFTs realise - they short sell to anticipate your liquidity premium (aka you want to sell NOW and you are willing to pay for it).
If you didn't want to get out so bad - you cannot be front run - because you wouldn't demand a liquidity premium.
Buy side liquidity means quick access to sellers at little cost and on favourable terms to you. This was always the case - HFTs just make it faster and easier.
Maybe there is a terminology glitch. Buy side and Sell side mean something else, generally. Buy side is commonly referred to people holding asset on book, and sell side are capital raisers or intermdiaries.
If I'm buy side, I want liquidity -- period. HFT does not provide liquidity, it provides decreased "viscosity". As you note, (observed) liquidity evaporates under high Vol. Which, if it were true liquidity, or if markets participants met the threshold assumpyions of EMH, would not be the case.
If you're trying to differentiate two sides to a trade on an exchange, that's usually referred to as Bid/Ask. Again, observed lack of liquidity on one side or the other (or: massive spreads), signify commonly held assumptions about the markets are askew.
If we throw away EMH behavioural assumpyions, and we throw away liquidity, what we are left with is the following:
(1) Opportunistic market participants;and
(2) Ultra-low transaction viscosity.
These are a shitty combination, from the perspective of public policy. The lack of viscosity actually increases the returns to increasingly obscure and opaque methods of market maniplation.
Prop trading (proprietary trading) just means a firm is using their own money as opposed to trading on a customer's behalf. So the retail investor is by definition almost always engaging in the practice.
I agree completely ... let the HFTs eek out a few pennies per trade (or even fractions of cents). But I also wanted to see how much bandwidth my 4G phone could really use. Sorry for the perturbations I might have caused and I promise I'm only doing what's good for man-kind. Now if I could just get my phone to make a decent call!
I don't know what you mean. HFTs don't do what's good for mankind - they do what's good for themselves. Just like everyone else. No one does what's good for mankind - they do what's best for themselves.
We allow those who don't harm others to continue doing what benefits them - until such a time as it is shown to be harmful - then we stop it. HFTs have not been shown to be harmful.
It's more nuanced than that. There are things like patents and copyrights. When it suits them, big C capitalists demand longer trading terms, then turn around and say the opposite when it comes to HFT (shorter trading terms). (They also do this with their accredited investor requirements).
HFT harms society because it cuts off the number of investors that can do active trading. It's net effect is the same as regulations the keep out entrepreneurs from innovating in things like health care and drones and car manufacturing.
Concentrating trading in a few hands is the problem we had - too big to fail. HFT will lead to barriers to new entrants (because of increasing startup costs).
>Concentrating trading in a few hands is the problem we had - too big to fail
No. You're confusing things. High frequency trading had nothing to do with financial bailouts. To my knowledge, no high frequency trading shop has ever been bailed out or deemed too big to fail.
>HFT will lead to barriers to new entrants (because of increasing startup costs).
Please explain this, how does HFT increase startup costs?
I'm not sure who you think benefits the most out of high frequency trading, but it's not huge banks like Goldman Sachs. My understanding is that the best high frequency shops are relatively small. They're made up of a mix of programmer and quants, not traditional investment bankers.
I'm not confusing anything. Yes the previous crash wasn't HFT but current trends will lead to the already rich (the banks) being the HFT's because the land surrounding the exchange is limited and costs will increase (HFT is about land and computer resources, and high speed networks - new models don't factor in as much). It won't be programmers calling the shots, they'll be employed by the banks. Already, the average geek is outgunned (overall competitiveness has decreased).
How do you figure the average geek is outgunned? There are multitudes of small HFT shops staffed mainly by geeks in Manhattan doing pretty well. HFT tends not to be profitable enough for banks to bother with, especially if you have to pay most of your profits to the programmers behind it so they don't leave and do it themselves.
Yeah because retail investors should do more with their money - not.
There's a reason these industries are hard and should be hard - these are serious industries with serious consequences. Health care - screw it up and you kill someone. Drones - screw it up and you kill someone. Cars - screw it up and you kill someone. Finance - screw it up and you lose the retirement savings of your investors.
These are not games to be played by unsophisticated people or green entrepreneurs. This does not mean that the extant incumbents are any good - it merely means that new players does not automatically confer innovation goodness (see natural monopolies/booms).
The argument you're making is in favor of my argument of regulating HFT (I want a limit to daily or weekly trade, others want more taxation).
As to regulation on healthcare/drones/cars etc. things aren't perfect, that's all I'm saying. I think things are too restrictive right now, I'm not arguing for abolition of all regulations.
edit: Yes, retail investors should be trading more. That's the idea behind the recent rise in crowdfunding. Innovation happens a lot faster in small c capitalism than in big C capitalism.
Crowdfunding will be the core of the next tech bubble peaking ~2016-2018 - run on afterburner with the success of product pre-purchase launches (irrelevant to investment), the aggressive lobbying of VCs, the credulity/unsophistication of most investors and the passing of the JOBS act combined with the cycling of alpha searching global credit saturating the niche industry known as Silicon Valley.
But I'll gladly take part and take people's money while the times are good.
I don't particularly care either way - I make money no matter what the market.
For many years George Soros has been suggesting that a tiny tax on transactions would eliminate a huge amount of high frequency bullshit (he was suggesting it for currency markets, but the same applies here). I think he proposed that the revenue generated could fund the UN... It's about the same Bill Gates's suggestion of charging a penny to send an email -- no impact on legitimate use and a serious impediment to spammers.
How does charging me somewhere vaguely in the ballpark of $36.50 per year qualify as "no impact"?
And I'm just an individual. What about when I send to an email list and it goes to 100 people? Should Google Groups pay a dollar to send out those 100 emails? What about MailChimp, do they have pay $200 when they send out a newsletter to 20,000 people?
I'm also concerned about how this money will be collected and how, from a technical perspective, all emails will be monitored. Every SMTP server will now have to be registered with the central oversight organization? VPNs and rogue email installations will be the enemy or blocked from communicating with anyone "on the grid"?
"A penny an email" is a figure that's tossed out, but it could be far lower. And let's ignore the technical aspects of how you'd go about collecting it (SMTP really isn't designed to make such payments easy, though we could bolt, staple, glue, and wire on yet another patch to the mess).
The point is that abusive email -- brain-dead bulk spam, and even annoying recruiter pitches and "real" business adverts -- are sent in far higher volume than even the all but the very largest of mailing lists. The standard figure bandied about for most of the past decade is that some high-90% of all email is spam. Let's say 97% (as Microsoft reports: http://news.bbc.co.uk/2/hi/technology/7988579.stm).
Spammers are generating returns based on a small fraction of a percent response rate on an overwhelming volume of mail, which is close to free. Sender costs are estimated at $0.00001 (http://www.clickz.com/clickz/column/2138759/make-spammers-pa...). That's 1/1000th of a peny per mail, or a cent per mil (thousand mails).
I suspect that raising that cost by even just a few cents per mil would be sufficient to put the brakes on most spam. For personal email, that would literally be a few pennies a year for even a high-volume correspondent. Protocols for mailing lists and other legitimate noncommercial use would probably rely on offsetting costs to users.
In reality, proposals such as hash-cash, greymilter, targrubing, and the like extract computational work from an untrusted SMTP sender as a requirement for accepting mail. It's a pretty effective way to put the brakes on high-volume delivery. Computational power is a pretty reasonable proxy for coin of the realm.
And if that's not enough ... there's plenty that's going on now with DKIM and other header validation that would serve to identify a sender. Extend that such that you don't accept non-signed mail (except at a very, very, very slow rate) and that you can associate a designated signer with a verified escrow account, and you've got much of the nuts and bolts of putting a system into place. Not that I see this as entirely desirable, but we may have to go there.
I suppose bitcoin could be used as the technical solution. Maybe receivers of money can white-list certain senders so they don't have to pay anything to send to them. Then only spammers and unknown people would have to pay.
Or the post office for that matter. There's actually one single organization that all those stamped envelopes get sent through, it's not decentralized.
Many exchanges want high frequency traders as market makers and pay them rebates for completed trades. High frequency trading is not inherently "bad". It's encouraged and desired by many exchanges.
Sounds as if the HFTs are initiating trades but not completing them.
In network terms, it's a SYN attack -- you're opening a port but not connecting to it, or dropping it.
A tax on uncompleted transactions might also help here. I'd say Soros's suggestion also has merits, though perhaps the incomplete transaction tax could be set somewhat higher.
Neither would have to be large to make HFT grossly unprofitable.
>A tax on uncompleted transactions might also help here.
I agree. My understanding is that regulations in my own country (Canada) are being put in place to mitigate such problems. I just think it is important to maintain the distinction between high frequency trading and high frequency quoting. They should not be condemned as one entity.
"Translation: the ultimate goal of many of these programs is to gum up the system so it slows down the quote feed to others and allows the computer traders (with their co-located servers at the exchanges) to gain a money-making arbitrage opportunity."
I thought the justification for HFT was liquidity. The market is clearly broken.
I believe that they were talking specifically about the programs which create orders which are never fulfulled. That way their programs creating real trades have an advantage (since they are running on servers located closer to the exchanges).
Bear in mind the exchanges aren't stupid, Nasdaq at least has instituted a penalty system for sending a bajillion orders and not actually trading [0]. One may certainly quibble with the exact rates and penalties but this is a thing that exists.
It makes a lot of sense to penalize that kind of behavior. Deliberately slowing down the quote feed or cancelling all your trades seems to be intended as a denial of service attack on your competitors.
So, the video made it sound like someone was holding a large amount of stocks and very quickly sold all of them (like the flash crash). It didn't lead me to believe that they were placing orders and pulling them before they could be executed. Was this type of order discovered after the video recording, or am I misunderstanding what they said?
Also, in the video they say:
"What you're saying is the individual investor has no shot. Mary and Joe sitting out there have no quants on their staff"
My assumption was always that the average trader holds their shares over a longer period of time, and thus aren't really affected by these HFT algorithms. Is that assumption incorrect?
No, it is not incorrect. The impact of HFT on buy & hold investors is probably a wash. Automated trading has cut the bid/ask spread but may contribute to volatility (it is tougher to say than it seems).
Either way, Mary and Joe were never competing with market makers and active trading strategies, and still aren't now.
1. From what I've heard, all large organizations need to trade with HFT strategies in mind. (Is this wrong?) So if Mary and Joe own index funds or mutual funds, then they are competing with active trading strategies. It is basically a tax.
2. If HFTraders are collectively making money, and presumably they must be, that money must be coming from somewhere. If, for example, the stock market rises 50%, via some sequence of gyrations, then there is only a fixed gain to be divided up. Maybe HFT increases the size of the pie, but that feels like mostly a higher-order effect.
That's wrong in the sense that it conflates trading strategies with HFT. It has always been a technical challenge to buy or sell large blocks; it is in fact one of the fundamental basic challenges.
The money HFTs collect tends to come from the bid/ask spread. It's money that was already being taxed out of trades, but now robots are competing for it.
That's not entirely true. I'm an amateur and I want to invest actively using statistics. But getting tick data (covering years/decades) is costly and dealing with it as well. I could handle it if it were daily (if all trades were limited to 9am auction every day. This would also allow room for much more extensive modeling of other data sources and longer term predictions).
That's not a very convincing argument. I'd like to participate in F1 racing, but the research and engineering costs are too high for me as an individual to partake in. If they limited racing to only 4-cylinder non-modded Hondas, it would allow me to compete in their game.
That's a false premise. The "purpose" of capital markets isn't to "serve society" any more so than the "purpose" of restaurants is to "serve food". Restaurants (and any other businesses for that matter) exist to generate profits. They merely satisfy that goal by serving food, but it could be done a million other ways.
The stock market exists not because of some idealized goal of serving society, but because companies want to sell partial ownership and transfer risk from themselves to the public in exchange for potential future returns on that investment. Anyone member of the public who purchases stock in a company should understand that agreement. It also has secondary benefits like providing liquidity for employees, etc., but the New York Stock Exchange wasn't founded with that purpose in mind. In other words, the stock market doesn't owe you anything.
That point aside, could you clarify how HFT hurts you? If your goal is to use stock markets as an investment vehicle, then your time horizon should be on the order of months and years, not seconds. If that's the case, then HFT has no impact on you whatsoever. If you do want to play in the second time range, then my analogy with F1 racing is perfectly applicable and there's no justification for your complaint.
This is a really dumb way of looking at it. Capital markets are a tool - they are a means, not an end. They are a tool society uses to allocate resources in a sane, equitable, and efficient way. To the extent they accomplish this (and they generally do a rather great job at it), they are a useful tool. To the extent they don't, we are justified in altering the way they function so as to achieve the results we desire. They are an organizational strategy, nothing more.
What they are certainly not, is some higher order of intelligence or ideal that we humans just have to learn to live with, or else. The stock market may not owe me anything, but it certainly owes us something, or we wouldn't use it.
That's not false - capitalism is justified by it's proponents by it's ability to serve the public better than socialism (the majority of the public wants more socialism - entitlements/public services etc). Some aspects of capitalism (copyrights, patents, HFT, pollution) are not serving the public and there are valid arguments from the opposing side.
My goal is to trade daily/weekly (in addition to monthly/yearly). I don't want to seconds/minutes/hours (and so can't many others - so the "competitiveness" of markets is actually reduced).
Ultimately, stock markets exist to raise funds for projects (exits for entrepreneurs, financing projects in large corps) - they occur on the daily-years timeframe, not seconds/minutes/hours.
You say HFT hurts the public, but you only show evidence that it hurts LFT. Why do we care about kow frewuency traders, aka parasites who profit from statistical patterns and not by contributing any value?
By that reasoning, HFT are parasites too. More generally, the stock market exists to allocate resources. Is society already so efficient that millisecond allocation needs to occur?
F1 racing is not a game, it is a major global industry. The top driver makes something like $40 million a year. Many of the advances we see in regular cars are designed and tested in those high performance vehicles.
But more importantly htf funds do not stop anyone from participating in the markets, it just harder and riskier to do so. Accounting firms had massive staffs with hundreds of binders and files for each client. Accounting software eliminated most of that. Were accountants talking about the unfair advantage that Intuit had, and how normal accountants couldn't make a living anymore? Probably. Welcome to the future.
HFT is not "automation" in the traditional sense. Stock markets exist to raise funding for projects (exits using ipo, new projects in public companies). Those occur on the day+ scale, not milliseconds. There's no point to increasing the speed of it (other than just for the sake of increasing the speed). It's like increasing the speed of the channel switching on your tv - after a certain point it's better to devote engineering resources on other aspects of the tv.
So if you owned a portfolio of stock and wanted to sell XYZ, you'd be OK if the market operated at a "day+ scale" and took 24 hours to come up with a bid price, and then if you didn't like that price you'd wait another 24 hours for the next price?
>That's not entirely true. I'm an amateur and I want to invest actively using statistics. But getting tick data (covering years/decades) is costly and dealing with it as well. I could handle it if it were daily (if all trades were limited to 9am auction every day. This would also allow room for much more extensive modeling of other data sources and longer term predictions).
It's not that expensive to get historical data. You can get daily for free. You can get minute data for years for ~$65.00/month and tick data for an extra $25.00/month.
The expensive part is direct market access, and even that is reasonable if you've got a successful trading strategy.
Well, the costs add up. You have to process that data as well afterwards. And all those resources are being diverted away from collecting other data and making longer term predictions. Multiply that by the millions who could be trading but don't (a few hundred dollars a month is a barrier to a lot of people, that might have been the profit of a small strategy that worked).
First of all, you haven't yet told me why high frequency trading is driving up data costs.
Secondly, if $100/month is stopping your business from being profitable, that's a fault with your business model, nothing more. It would cost more to get a medium size Windows instance on Amacon EC2 for the month.
>Multiply that by the millions who could be trading but don't (a few hundred dollars a month is a barrier to a lot of people, that might have been the profit of a small strategy that worked).
I don't think you can claim that millions of people are being locked out of the market because of data costs.
If there's more data, then it's more costly to deal with it - simple as that.
What if there's a little appliance that you plug in which makes predictive models (you can think of it as installing software on your laptop). What if millions of people want this appliance but it costs a $100 a month but could have made about $100 dollars a month? They will choose not to invest.
This might seem contrived, but it's also the scenario behind the web (lot less blogs when it cost $100 per month).
The net effect is that less predictive models of the economy are created. This is bad because that's how capitalism allocates resources.
The larger framework is that competition increases quality. Any barrier to business decreases competition.
>If there's more data, then it's more costly to deal with it - simple as that.
You can still trade on daily bars. Anyone can. Whether you sum up a days worth of data into a daily bar, or you have an auction once a day you're still going to have the same sized data set.
>The net effect is that less predictive models of the economy are created.
No it's not. $100/month is a reasonable cost. I don't know how to make this clearer to you. Data costs are among the CHEAPEST part of the equation when you're building a financial model. Quantitative analysts are paid six figures. Skilled programmers are paid on the order of six figures. Getting data costs down to $100 is not going to make someone go "Oh you know what? I'm ready to put in 100 hour weeks developing financial models because I can now afford a bus pass".
The examples you're giving sound ridiculous because they are ridiculous, and so is the premise you're basing them upon. It's like me telling you that I want to become a programmer but a $100 laptop cuts into my expenses too much.
>The larger framework is that competition increases quality. Any barrier to business decreases competition.
$100 will not increase competition. I can't put it any more plainly than that.
Well, if you want to brush everything off like that (including how you make predictive model from daily data when there's so much volatility when you actually want buy/sell) then you're clearly trying to win some type of "argument". Have at it boss.
>Well, if you want to brush everything off like that (including how you make predictive model from daily data when there's so much volatility when you actually want buy/sell) then you're clearly trying to win some type of "argument". Have at it boss.
Trade on opens. Occasionally there are gaps, but they are traditionally due to big news, almost always related to fundamentals. If you're modelling the stock market as a random walk, then you're just as likely to have volatility go for you as against you under normal market conditions.
You should be upset. Why is it that only select few can have reserve requirements and make interest on free loans from the central bank. It's a direct transfer of wealth to the chosen few.
Is it as easy as possible? Can you borrow straight from the central bank at the low interest rates? The more regulations/expenses you put in the way the less competitiveness there is and the more unfair it is (hence giving free money to the powerful). (Also, will I get a bailout?)
HFT is an exploitation technique to extract value from the market. It drives away capital formation and investor participation, and is bad for everybody except the HFT traders themselves. All non-HFT traders are affected negatively.
This makes Mark Cuban's article on High Frequency Trading (HFT) even more relevant than it was before (link: http://blogmaverick.com/2012/09/21/what-business-is-wall-str...). WHY is it not regulated or at the very least taxed, when it creates such outsized risk for the public and doesn't accomplish anything for the market besides increased volatility
This sounds like "don't go outside if you're worried about being mugged". Sorry, no we almost all pay taxes used for bailouts and have our retirements invested there by our employers. We get to criticize and advocate for change.
The NANEX research page[1] has a lot more information about the different MOs that they spot. It looks like NANEX first spotted it on October 1st:
"On October 1, 2012, we detected a new form of quote stuffing that tries to hide under the radar. It involves hundreds of stocks and millions of bogus quotes during the trading day. The algo that generates them is very careful to spread out the stocks targeted and limits each test to a blast of exactly 200 quotes in 25 milliseconds or less. The quotes all come from Nasdaq, and sometimes affect the Best Bid or Offer: the only change appears to be a fluttering of the bid or offer size."[2]
Doing my best tptacek impression "the nut graf is":
"We believe that this algo will continue to grow and if left unchecked, could very well contribute to the next flash crash because it removes precious network capacity and provides zero economic value such as price discovery."
They posted an update on the 5th with a neat visualization of the trading activity[3]. If you just want to see a neat animated gif the direct link is:
The second and third fields have the following possible values:
13 4 N P Q
0 1 2 3 4 5 6 7 8 9 a A b B C D E F
I was looking at the nxcore api docs [5] and I could not tell what these fields could be. The field values do not seem to match up to the listed values for exchanges or order types.
I am delighted and rather surprised to say that someone from NANEX responded to my inquiry about the second and third fields.
"P = ARCA (we've since determined that the algo only ran on NYSE and Nasdaq listed issues)
N = NYSE
Q = Nasdaq
The 3rd column is the SIP multicast line that would carry the quote. For
Nasdaq listed, those are carried on UQDF which uses 6 lines and we label
as A, B, C, D, E, F. For NYSE/ARCA listed those are carried on CQS which
uses 12 lines, we label as 1-9,a,b."
Excellent write up, I was hoping Nanex had some info on this. Tangentially related, The Rise of the HFT Machines is another great Nanex article that argues how it's not so much HF trading that threatens market stability, it's the HF quoting[1].
There are some great charts illustrating how every time exchanges increase quote processing capacity the HF algos gobble up the new bandwidth, but ultimately the number of executed trades has remained relatively consistent over the past five years.
>>> The intent of these players isn't necessarily to slow down an exchange, but also to force "opponents" to slow down as they handle more of a load as the orderbook is frequently updated with garbage orders.
LOL. You can't do that. It is illegal. And if you'd do that intentionally, SEC would be calling you the same day.
You can't be serious. Quote stuffing the channel is standard HFT practice. 99% of the orders placed are never meant to be hit. The SEC is incapable and uninterested in policing this stuff.
Put a .5 second minimum TTL on every order and you'd see all the 'liquidity' provided by the HFT world dry up instantly.
You typically get throttled by the exchange when you send too many messages. When your messages are rejected, you monotonically lose money until you can message again. For this reason, I highly doubt anyone is sending an uncontrolled amount of messages to any exchange on purpose because surpassing messaging constraints is unprofitable, which would violate everyone's primary motive.
The orders not meant to be hit is a separate issue from the messaging. These orders are there because of
1) quoting requirements designed to mitigate volatility,
2) to give traders the feeling of depth (illusory or not) because market participants tend to interact with exchanges that look like they have thicker books,
3) to gain queue spot due to any FIFO component of the exchange's matching algorithm in case the price moves to the level where these orders could get executed,
4) to gain order allocation due to any pro rata component of the exchange's matching algorithm.
All of these reasons are controlled by the exchange and to some extent the SEC. The first 2 reasons are the result of the exchange trying to make money by attracting participants. The last 2 reasons are the result of participants rationally reacting to their incentives as dictated by the matching algorithms designed by the exchange.
Right. My point is it is illegal to enter an order into the market with the intent of goading someone into trading or to manipulate the price unless it is a bona-fide offer to buy or sell. Intent matters with regard to illegality, and I don't think anybody questions the 'intent' of most HFT shops.
Creating the illusion of volume where none exists has long been illegal - people used to paint the tape long before HFT exists to achieve the same thing.
You've provided a good description of why HFT do what they do, and one could argue that laws need to be changed to allow this market behavior. (I would disagree) Much of what they do is illegal by present law, but none of the big market players want the law enforced, so the SEC looks the other way.
The illegal activity you're referring to are the bluffs? I'm under the impression most people have not successfully automated bluffs although I have heard of people manually doing it and using algorithms to exit or hedge. In my experience, most HFT does not intend to manipulate markets, maybe solely because this is difficult to automate and there's a lot of lower hanging fruit (not because they are altruistically abstaining from such behavior). It's difficult to automate bluffs because most HFT relies on backtesting and it's very hard to backtest high market impact strategies. In contrast, it is my impression manual traders can get very good at market manipulation.
There is a lot of shady stuff that goes on in trading due to conflicts of interest. Anyone intelligent or informed enough to know what's going on is financially incentivized to be secretive about it. I haven't gotten the impression the SEC is intentionally being incompetent- they really ARE just incompetent because anyone smart enough to realize what's happening does not join or remain in the SEC. Incompetence is the simplest explanation for the crazy rules and fines they have enforced, and the rules obvious to actual traders that they have ignored.
The whole reason for quote stuffing, co-locating and competing on latency advantage is because the exchanges only accept quotes at one-penny increments and process them FIFO (first-in first-out). The restriction to one-penny quotes was relatively recent, before that the minimum increment was 1/8 of a dollar.
If the exchanges accepted quotes at fractions of a penny (for instance float values), then the speed/latency of quotes would take a backseat to price. It would make pointless a lot of the current shenanigans.
I think many exchanges actually already do this, with a rebate provided for actually orders that are executed. The idea being that it is fine to cancel five orders for every one fill or some other ratio of cancels to fills. Check the pricing for trading with Interactive Brokers. I think they are passing on these exchange fees directly.
How does the exchange charge for access? Per trade, per bandwidth, per colo distance?
It would be simple for the exchanges to reign in trading by adjusting their pricing model. Make it more expensive to make each type of request. Adding a few cents to the request would squash HFT while not drastically effecting other trading systems/people.
Translation of
"Translation: the ultimate goal of many of these programs is to gum up the system so it slows down the quote feed to others and allows the computer traders (with their co-located servers at the exchanges) to gain a money-making arbitrage opportunity."
Annomyous is (D)Dosing the market. :D
I suppose that's technically correct-- we don't know how the algorithm works. But they make it sound like they have no idea to whom the algorithm belongs. If your logging can't get you this information (the bot's owner), I have no hope for the security of the exchanges.
If this kind of thing is a problem, then read your logs, visit the 'offending' party, and Do Something About It.
Read your logs? The people that identified the algo are just looking at quotes coming in from different exchanges. What logs do you think they can look at?
So, requests come into the trading system and there's no tracability? No credentials? No security? I'm not very religious, but heaven help us if this is the state of the Wall Street automation!
Each participant is identified to the exchange (or ECN or ATS) by a MPID (market participant ID), but individual quotes on an exchange's limit order book are not tagged.
Another reason to limit trading to daily or weekly or montly. The trading arms race is focusing on speed when it should be focusing on predicting longer term outcomes. The easiest way to do so would be to limit all trades to once per day in an auction system. Then traders would pour their resources into building sophisticated systems that make long range predictions and so also identify new innovations/companies that should exist in the market. This is in literally almost everyone's best interest - investors, public corp's, governments, consumer - only a small group of high-speed traders would disagree. Why this hasn't occurred yet is mind-boggling.
Suppose the NYSE has a new rule where trades only happen once a day, in a morning auction. Imagine that you just bought some stock at the auction in a certain company. Later on in the day, the company announces some catastrophic news, but you have to hold on to your stocks until the next day's auction, only to suffer great loss when you sell.
Meanwhile, on a options exchange, a different exchange, where trading happens throughout the day because there is no such regulation, other investors in the same stock get to hedge their position by selling call options. They get to act ahead of you because their exchange has no such regulation. Unfortunately for whatever reason, you have no access to this venue. Should regulators regulate these exchanges as well? If they don't, someone will complain that it's unfair to people who have no access to such exotic venues. So, let's say they regulate all exchanges in the country.
Then, people who really want to trade in the middle of the day have to do it the 'good' old-fashion way, away from the exchange, making phone calls to each other, through brokers, where there's less transparency and more chance for corruption and unfair deals because the fair price of an instrument is unclear due to the absence of exchange activity. This would be a step backward.
Additionally, in terms of raising and reallocating capital, companies whose stocks are thus regulated will be at a disadvantage relative to their international counterparts whose stocks are more liquid in the absence of similar regulation, as capital tends to flow where trading is more convenient. So, this effect would impact the country's competitiveness as well.
Bonkers, currently exchanges are only open from 9:30am to 4pm EST Mon-Fri. So your "doomsday" argument is equally applicable - most companies make announcements outside trading hours and most unhedged investor loose money once marker opens if announcement is negative.
Typical US equity trading hours roughly coincide with normal working hours. Not many people would want to stay around too late. Even if they did, the liquidity they need may not be there because most people have stopped working for the day.
If that's the case, things are so efficient that things will spill over in those ways, why not open the exchanges to everyone. Anyone should be able to list a company for zero cost, and anyone can purchase stock in anything for zero cost and no net worth requirements. The competitivness of the market will sort it all out.
Unfortunately, Nasdaq and NYSE are not charitable organizations. They are for-profit companies. Why don't you go ahead and make history by creating the type of exchange you described? I don't see how the law would intrinsically stop you, though the economics of running a company might.
The evil Rothschild's are still costing today. A redistribution of wealth that was accumulated prior to 1900 should be instituted (or maybe for even before 1980).
High frequency traders talk about this and I heard about one being developed. Regulators and basically everyone you listed don't know what's in their best interest so it's up to the traders who decide they want to stop the arms race. It's hard to switch over bc traders go where the liquidity is so we'll have to see if they can overcome the chicken and egg problem.
I didn't say long-term investment. I said limit trading to once per day or week. Then resources will go towards creating predictive models in the longer time range than milliseconds/seconds etc. Even limiting to monthly would be good. Most real world projects (that directly benefit soceity) occur on days/weeks+ timescales. There's no reason for stock trading to occur faster than that since it's ultimate purpose is to finance those projects.
Historically buying into an index fund during a crisis is a perfectly reasonable way to achieve relatively easy alpha above the mutual fund/bond rates (~12% per year).
This type of investing (aka diversified value) is completely separate from HFT. As far as many retail investors should be concerned HFTs can duke it all out between themselves as much as they want - it really doesn't matter and is irrelevant to your long term investment decision.
Indeed HFTs provide the benefit that during times of crisis they provide plenty of buy side liquidity for you to get in at a lower spread and a lower spot price relatively easily.