This isn't much of an issue for traders That's a nice benefit of having 40+ trading venues in the US. They had problems at the open as well with connectivity on one of their gateways.
Issues like this crop up all the time and most of the time they are resolved before the open.
The good news is that they aren't reporting any lost trades or trade busts yet so this isn't as bad as the BATS open at BATS:)
Having said that, they've announced that they will cancel all open orders, that is a huge deal, I can't remember the last time they did that.
To put that into perspective, cancelling all open orders would be the Silicon Valley equivalent of Ebay loosing all bids on their current auctions.
NYSE has always been considered second rate in their IT compared to the NASDAQ and BATS and this won't do much for their reputation.
EDIT meant BATS not FB, thanks!
One other point to keep in mind when throwing around hacking conspiracies. The exchanges aren't running on public networks. You can't DDOS them or hack directly into the matching engines. Though I'm sure you can break in via some other NYSE owned network and make your way to the matching engine somehow.
To hook into the exchange you either go through a blessed intermediary like GS or you plug directly in via colocation. You just can't keep pinging the NYSE on port 80 to bring it down.
Actually, the FB IPO open fiasco was on Nasdaq not BATS. Nasdaq, has had way more serious issues in the last 2 or 3 years that I can recall compared to NYSE.
[EDIT] The BATS issue you may have been referring to is BATS IPO when they tried to list on their own exchange. The technology failed miserably that day, so much so that they decided to pull the IPO altogether. One of the biggest fails I can think of.
I'm really happy to see that they're canceling all open orders. It's much better than letting them stick around and then execute with unexpected behavior. It looks like NYSE learned from what happened related the FB opening (just as you alluded to), and what happened with Knight around the same time. Playing it safe can be a smart move, even in the stock market.
My immediate reaction is that their system shouldn't require canceling all open orders. But I admit that I don't know much about matching engines.
If they roll out a new version of the module that creates orders, I would think it makes sense to include the version of this order module in the order data structure. This way they can delete only the bad orders, created by the faulty module version, instead of all open orders. But it's also possible that I'm way in over my head here :).
Yes in the sense that about 90% of hedge funds just use a SOR(Smart Order Router) from their executing broker. So from the funds perspective they are just sending an order to GS that should execute on the "US Market" and they expect GS to send it to the appropriate venue.
Most funds just trade through an intermediary like GS and all intermediaries have SOR's that route to each venue in the US as per RegNMS
The remaining 10% are your market makers and HFT's and for them this is a bit of a headache but it happens and they are well equipped to deal with this. No fund that I know of is a market maker for just the NYSE, but if there is then I guess they just go home early?
right. one trading head just noted that it's a "non-issue." the timing is interesting, though. it's midday so there's lighter volume. big problem is going to be with index calculation. other exchanges are functioning just fine and other venues are seeing plenty of action and stability.
I get the impression they'd heard on the wire that something might go down.
Given the blame coming out of China, it wouldn't surprise me if they tried to crash the market on opening with a flurry of sells. Looks like NYSE got a whiff of the pending trades and the NSA stepped in. All trades dropped. Crisis averted. For now...
It should be noted that this concerns only the regular NYSE and NYSE MKT (formerly known as AMEX). Their ECN NYSE Arca is working properly. So, this is really not that big of a deal for the market.
This doesn't mean all trading is suspended, it just means you can't trade specifically on NYSE or NYSE MKT (formerly Amex). You can still trade NYSE listed stocks at all the other exchanges (BATS, NASDAQ, NYSE Arca, etc). Obviously a black eye for NYSE but it's not as big a deal as the media are making out.
It's still pretty bad. Imagine being the IT dept at the NYSE right now. The trading is likely being rerouted to other exchanges. The Wall Street Journal website is also down.
Racial comments are blaming statements. I've read plenty of those on here over the years. Here's another example of fsk making a blaming statement which isn't racial in origin:
> You have to be on Facebook if all your friends are on Facebook.
Given I canceled my Facebook account in 2010, shortly after I was front and center in the news about tech addiction, I can definitively say that I don't have to be on Facebook. I also can say I can't speak for you on that matter, zabcik, or YOU, all you wonderful HN peeps. That's up to you guys to decide for yourselves!
I was considering the impact a 'blaming statement' button next to each comment might have on the site. It possibly could help curate and educate the posters on here about the negative impact they have on conversation and trust. Sometimes the backlash to a blaming statement is worse than the statement itself.
BTW, I get the sense (observation) from fsk's comments that he/she might be having a bit of trouble with their career, which might be why such a statement is being made. If so, I hear you fsk (I'm a 48 year old programmer/evangelist myself). Just remember you can't blame others for your own feelings! :)
I know a disproportionately large percentage of Google and Microsoft are Indian too, quite likely Amazon as well. Making the connection that the NYSE glitch is somehow connected to the tech staff being Indian is absurd and completely racist.
I took the implication to be that NYSE (like some other companies) makes disproportionate use of H1B and similar. One purpose of making very heavy use of H1B is to get workers who are more desperate and likely to accept worse pay, conditions, and longer hours. This is an open secret at at least one of the major companies like the ones you cited. The "prevailing wage" requirement is not hard to work around.
Hacker News might not like or expect this, but if you run a sweat shop with little regard for the welfare of your employees, you should expect more mistakes and lower quality output. This is not a comment on the racial ability of people whose ancestors were on the Indian subcontinent. There is no evidence I ever heard of for any inherent difference of ability by race. But sweatshops, that is another story.
I work at the exchange...there's no off-shore sweat shop. There are no people in india running commands or coding for 90 hrs a week. Folks in multiple US cities fixed this problem, including myself. I've only read comments here previously until I saw all this speculation. Its funny people are so adamant in their theories despite no evidence besides circumstance.
It was a complex IT problem (no I can't get into more details) that we fixed. No hacking attempt, no conspiracy with the United Airlines Issue or WSJ...shit breaks.
I wonder if people are looking for a 'deeper' story because "shit broke, fix was prompt, PR was processional" isn't as interesting as sweat shops, foreign hackers or cover ups.
You are in the know, because "you run a sweat shop with little regard for the welfare of your employees"?
For what little it is worth, I've found H1B employees to run the full range of amazing <-> incompetent at rates indistinguishable (by me) from "natives" in the Bay Area.
I posted above....it was a complex IT problem, fixed by individuals in multiple US cities, by US citizens. No hacking attempt, no Wall Street conspiracy...computers just break, and today they broke more than normal.
Btw, i'm purposely being vague since this isn't really public information. All i'm here to say is we don't run sweat shops, this wasn't a hacking attempt, there is no relation to the issues at United Airlines and WSJ...and last but certainly not least...
It's fun to guess at what's happening but folks should be shunning people who present their ideas as "self-evident" without presenting any actual evidence. Since when do we take people's word for it on the net anymore with a "source"?
I'm confused why it is in your interest to create a new account to post on here with a report that it's an internal issue, but then offer not even the barest technical detail when you apparently have evidence, all the while chiding others for having no evidence themselves.
Your vagueness isn't serving anyone. If you're worried about confidentiality, you would have been better off not posting at all.
The only things that gave me pause today in attributing this all to an Occam's Razor explanation are:
1) The government's careful wording about this not being an "intrusion" (which leaves open DDOS).
2) The lockstep tempering response is about what I'd expect from an industry which, almost as a matter of national if not global security, has to project a stabilizing confidence to averts irrational panics over even mundane hacking attempts.
I can believe that it is a simply an internal IT issue (apparently requiring people in multiple cities to work like crazy), but I also wouldn't be surprised to hear a completely different story emerge in the next few months as they slice the sausage of public release. It wouldn't be the first time that the initial narrative of a rogue element in the financial industry was shown to be inaccurate, and only revealed months later after an appropriate cooling off period.
> Your vagueness isn't serving anyone. If you're worried about confidentiality, you would have been better off not posting at all.
IMO 'thetruther's putative view from the inside is useful even without supporting evidence. Not every assertion needs peer review, and the community doesn't always have the right to demand it. (That doesn't mean I'd make a significant decision in reliance on his unsupported comments, of course.)
What in particular do you think is wrong in the GP's comment? I see a bunch of pretty much self-evident statements about quality of work produced in sweatshops.
The technical glitch is probably related to China news somehow, but it is rather meaningless for trading in general. NYSE has pretty antiquated technology compared to other exchanges.
As the saying goes, never attribute to malice what can be explained by incompetence.
In the case of United Airlines, be prepared to adjust your threshold of expected incompetence dramatically upward; since their merger with Continental multiple years ago, system-wide outages have been an every-couple-of-months thing.
ZeroHedge was down for a bit as well. I thought maybe it was an AWS (or other cloud provider) issue causing all the problems, until I read about the NYSE issue.
Maybe there's a stock market hacker flying around on planes...kinda like in the movie Hackers. Of course, in that movie they had no trouble cuffing him without grounding the plane. :)
interesting question and i'm not familiar with the rumors. is it that the shanghai index crash is synthesized? not being completely fluent in the financial sector, how would this be achieved?
Financial markets can be manipulated provided you have the right keys. At this point in time, this is just a wild conspiracy theory and it's explained as retaliation for all the cyber attacks allegedly carried out by the Chinese govt or people affiliated with it.
There's plenty of reason to believe China was in a bubble long before these last couple of weeks, and if you don't like somebody and want to do them the maximum damage, you want to help inflate the bubble, not pop it. (Well, at least, if you have a rational view of how markets work.)
We all knew today would be "bad for wall street". The Chinese stock market is crumbling (due to economic, not technical reasons) and Greece is in the fiscal equivalent of the crazy guy barricaded in his house with this gun collection.
"A trader on the floor of the exchange in lower Manhattan, who spoke on the condition of anonymity, said that after the suspension began, traders were told that the problem was related to updated software that was rolled out before markets opened on Wednesday.
According to the trader, the exchange said that the new software caused problems soon after trading began on Wednesday and the exchange decided to shut down trading all together to fix the problem.
A representative for the exchange did not respond to a request for comment on the trader’s account."
Seems like they were having issues earlier today, since before market open. Looks like a network issue based on the previous reports. Those were marked as resolved; unsure if the current issue is related to the previous but it seems likely.
Other exchanges and trading venues appear to operating normally.
Well, one thing this has done is bring the paranoid out of the woodwork. I've rarely seen such a display of tinfoil hattery on HN as I have done in the last few days.
this is nothing compared to the comment thread on zerohedge. they're connecting jade helm, china, greece, and this into something pretty extraordinary.
Yeah, but that is Zero Hedge's audience though: the upper echelon, rarified air, pure-bred market conspiracy theorist. What surprised me is seeing it start to creep in HN in such a volume.
And occasionally it still can be - the odd nugget of outside analysis and opinion from anyone but "Tyler," mainly. Don't get me wrong, I enjoy Zero Hedge these days too, but mainly for entertainment value than anything actually insightful.
Very interesting, obviously matters very little to retail traders as most retail brokerages either sell order-flow to marketmakers (e.g., Ameritrade) or have their own smart-router that looks at the liquidity of all exchanges/ECN and decides how to route their customers' orders (Interactive Brokers).
Quick question for trading peeps out there, is there a reason why one would want to direct orders directly to NYSE? Is it because it's still the place to trade bulk orders? (vs. say BATS or ARCA).
Combining this with the United airlines "glitch" -- it is definitely suspicious. What are the odds of two high profile failures happening at the same time?
Sounds like a similar problem to the birthday paradox - only instead of looking at the likelihood of any two people in room having the same birthday, we are asking what is the likelihood of two major companies having the same major, visible failure day. There is a nice explanation and calculator here:
The parameters that you provide would be the number of companies that would be newsworthy and the number of possible "dates". You could probably back out the latter from an expectation on reliability (five nines, as an example) or think of it as the time between major, visible failures (e.g. enter 3650 if you think of it as a once in ten years kind of failure).
If there's smoke there's fire. In my view, Occam's razor might indicate that coincidence is unlikely, and that the two major incidents in US on the same day China's market crash suggest it's a state sponsored cyber attack.
> In my view, Occam's razor might indicate that coincidence is unlikely, and that the two major incidents in US on the same day China's market crash suggest it's a state sponsored cyber attack.
Occam's razor calls for an explanation that does not require the unneeded introduction of a new variable. If you need a state sponsored cyber attack as the alternative explanation, the coincidence theory should be preferred.
> Cyberattacks originating from China is not a new variable
Sure, but its mere existence in other contexts does not automatically make it the cause of the NYSE etc. glitches. The assumption of causal relationship without any proof is what violates Occam's razor.
(And that map, while pretty, does not proof state sponsored cyber attacks. If I rented myself a $5 VPN account to Russia and then nmap whitehouse.gov, I would appear in that map as an attacker from Russia.)
China's market has been crashing for a few days now. How does a cyber attack on the NYSE help China? All trades will go through just fine on NASDAQ and BATS.
Occam's razor would actually suggest that coincidence is much more likely explanation than a malicious hacking by a (especially particular) foreign state actor. You have to look at relative probabilities.
You state this in response to a comment linking to a good description of why, precisely, this is broken thinking. Did you even read it?
Also, you've got Occam's razor around backwards, always a good way to cut yourself in public (Occam's razor resists adding more complexity to the solution without evidence)
"NBC News quoted two unnamed U.S. officials as saying that there is no indication that the market shutdown was related to the computer problems that temporarily grounded all United Airlines flights Wednesday morning."
wsj goes down, nyse goes down due to technical glitch. seems like a crazy coincidence or some very large state who have a track record of infiltrating and disrupting America electronically at a time when the leaders of that state is feeling the heat and need to stop speculators.
so you're saying that china has initiated some sort of half-cocked single-exchange attack? all other exchanges are operating, and trading is maintaining volume, and asset prices are stable. not sure i'm seeing the reasoning for such an attack.
Interesting point. Every 7 years, all the Jewish farmers in upstate New York and Long Island have to either leave their lands fallow or lease them out to non-Jews. Clearly that's what's shaking up the market.
In all seriousness, how are you supposed to "refute the central point" when the central point appears to be an anti-semitic conspiracy theory? How do you construct a reasoned, evidence backed counter-argument to an absurd, evidence-free belief? How do you prove a negative? (How do you reason a man out of a position he didn't reason himself into?)
How do you construct a reasoned, evidence backed counter-argument to an absurd, evidence-free belief?
One answer is that you fight stupid with stupid. See for example, Ali G winning an argument with a creationist by asking him, "OK, has you ever eaten a banana?"
But, on HN, ignore and downvote. Stupid isn't welcome.
This is what happens when you pay ENGINEERS less than 100k in NYC! This is what happens when you hire a bunch of H1Bs! This is what happens when you don't invest in tech infrastructure!
Why aren't Vladimir Tenev and Baiju Bhatt being acquihired? Why isn't David Byttow making 1 million plus as a full stack engineer for the NYSE? It's because companies, even those heavily dependent on their systems, see tech workers as a cost instead of THE business.
This will more than likely never change. Underpaid engineers vastly outperform their salaries. But when things like this happen I can't help but feel a little glee.
Issues like this crop up all the time and most of the time they are resolved before the open. The good news is that they aren't reporting any lost trades or trade busts yet so this isn't as bad as the BATS open at BATS:)
Having said that, they've announced that they will cancel all open orders, that is a huge deal, I can't remember the last time they did that.
To put that into perspective, cancelling all open orders would be the Silicon Valley equivalent of Ebay loosing all bids on their current auctions.
NYSE has always been considered second rate in their IT compared to the NASDAQ and BATS and this won't do much for their reputation.
EDIT meant BATS not FB, thanks!
One other point to keep in mind when throwing around hacking conspiracies. The exchanges aren't running on public networks. You can't DDOS them or hack directly into the matching engines. Though I'm sure you can break in via some other NYSE owned network and make your way to the matching engine somehow.
To hook into the exchange you either go through a blessed intermediary like GS or you plug directly in via colocation. You just can't keep pinging the NYSE on port 80 to bring it down.