Wow... Being down once - OK, it was record-high volume, servers were strained. Twice - well, it was only a day after the first time and it takes time to fix these things. But a third time, nearly a week after the first incident (and once again on a volatile day where users stand to gain/lose a lot of money)? Really hard to think of any acceptable justification for that.
The simple answer is that the reason the system failed the first time wasn't easy to fix in the short term. Either they need to acquire more hardware resources or redesign some system so that it scales better. Not everything can be fixed overnight. Take a look at reddit - that's a website that collapsed repeatedly for months and months and months. Sometimes it's not something you can just 'fix'.
I'd argue that the consequences of reddit not being available for a certain amount of time are absolutely not comparable with a financial services' firms infrastructure.
Mistakes, any mistakes, are very hard to excuse when people, or other financial entities happen to lose money.
Comparing it with a site like reddit is, frankly, a bit disengineous.
Consequences, no. Infrastructure failures, absolutely. What's being hosted isn't as important when it's broken and you're struggling to fix it. Sure, when it's a financial service's infrastructure, you're worried about other peoples money, but it's still something that's broken and may take time to fix.
The load testing you need to do for trading is considerably higher than any chat forum to go live .
Your customers are paying for speed and performance .
In a industry where users spend millions to be closer to the exchange if you are not provisioning for some multiple of your peak load you are going to get slaughtered in the market
I don't think that argument really holds up when Robinhood is the subject. I mean they still don't even have phone support or same day guarantees covering them responding to your email.
Robinhood is nothing like Reddit. It has a much bigger valuation, more resources, strict regulations to follow, and the responsibility of safely handling money for millions of people. This should be fixed within days even if it means redesigning the entire system.
The last blog post blamed overloaded DNS so it doesn't seem like a major architectural problem, which is why it's surprising that they're down again.
The last 20 years I have been working in heavily regulated environments; in cock-ups like this were your Retail clients got royally screwed (for a second time in such a short period), on a repeat issue, the hammer will fall HARD on their heads. Anyone wanted to sell to minimize losses has been severely damaged.
There are many ways to resolve these issues FAST. The easy way is to throw money at this. Go tomorrow (literally) and bypass all procurement controls and go to a mega big provider and scale this asap. Any financial services with a half decent IT has done the paper exercise to this scenario (at least for the purposes of BCP/DRP). The slow/better/mature way may be too slow, especially with the current market conditions.
The RH folks will definitely get a visit from SEC, their external auditor, and their external auditor will get a visit from SEC.(their auditor will be in the deepest of shits)(how come they failed to spot such a going concern issue?)(what the hell were they looking for on their audits?)(did they only send juniors over there?)
I feel sorry for the retail traders that got knocked down. I think that anyone locked in buying at 27-28k (US30) should wait 6 months to breakeven and after the US elections (irrespective of the winner) there will probably be a rally.
They aren't that fantastic either. They had issues today as well, I couldn't exit out of spreads, so I had to liquidate everything and that was failing too. It was a hot mess.
Edit: I didn't actually want to liquidate all my holdings but it was better than staying in and risking everything else.
"redesigning the entire system" was an exaggeration. They have a serious problem if that's really what they need to do.
Rolling out fixes fast, even if they require intensive changes, is completely reasonable and expected in many industries though. They should have the talent and procedures to get it done. This isn't some startup web app, it's a multi-billion dollar broker managing millions in client funds. It's bordering on incompetent to have 3 outages in 2 weeks.
While I don't agree with OP about replacing an entire system overnight. I do remember a friend who worked on a trading floor, and when they had bugs, their manager would say "the traders are taking a 30 minute lunch, you have that long to fix it" with the clear implication that they'd be fired if they didn't.
So I'm not sure if I know that the state of the art for trading platforms is as rock solid as everyone is implying, and Robinhood seems to be way far off from whatever gold standards there are, see infinite leverage bug. So I don't think it's crazy that they move quickly to fix it.
I'd never be able to stomach the pressure, and I wouldn't wish it on others, but it doesn't seem crazy.
I understand the risks need to be weighed against restoring service for their users who might be losing money and avoiding regulatory fines.
Changing a single line can introduce a different bug. Use proper QA and testing to catch as many as possible as with any development.
My emphasis is on getting things fixed quickly. They need to do whatever it takes to get systems online asap. Not sure what's so controversial about that.
Well to be honest we don't know how badly things are broken only they do. But the concept of moving fast while doing things perfectly is the holy grail of software development and not as easily achieved as you seem to make it appear.
I only said it needed to be done fast, regardless of how much work. Of course it's not easy.
I'm surprised by all the misinterpretation in this thread. Seems like it reflects the laid-back West Coast/SV attitude that isn't a good fit for high pressure time-sensitive work in other industries.
You seem to think (or at least imply) that hard things can be done fast if only you work hard enough at it, that it's just a matter of trying. This is just not true. Not sure how else your posts should be interpreted. I've spent days with a team (yes a competent team) just tracking down a bug, let alone fix it (although usually when it's tracked down it's relatively quick to fix). If this issue involves multiple systems in a highly complex environment, then it very well could take a while to address fully, no matter how hard they work at it.
Because that's usually the case. Crunch time, disaster recovery, and emergency fixes are common in every sector from video games to aerospace. If you can't fix then switch to a secondary, or rebuild from backup, or throttle users, or process manually, or do anything other than be completely down.
RH wasn't prepared with any contingency. They should have a resolution for their users - even if they can't find or fix the original cause. That's the failure I'm talking about.
See the 2 other users in this thread that describe similar high-pressure situations.
Depends on what the issue is. Could be something that can be quickly fixed or not. Although going by the lack of a resolution I’m assuming it’s not.
Reality is, without knowing more about what’s causing this it’s impossible for either of us to say. If there is indeed some fundamental bottleneck that was previously not known, then I certainly won’t be surprised if it takes a while to sort out.
Now you can say they should’ve load tested, capacity planned etc etc. But we are where we are. Still can’t go back in time to turn this into a quickly fixable problem if it’s currently not.
Edit: also pretty disappointed that we don’t know more about the root cause. As an user I’d want to know what the issue was and what they are planning to do to about it to evaluate if I should trust them going forward.
> needed to be done fast, regardless of how much work
This is the part you don't understand. There is a difference between digging ten one-foot deep holes vs one ten-foot deep hole. People need time to plan how to coordinate and then get on the same page so that everyone can work at their own pace. That is the part that is not parallelizable and is the rate-determining step.
The context is all lost here. Plenty of other companies and industries have emergency action and disaster recovery. People don't work at their own pace, they work to the deadline with solid procedures. They can fix and replace entire components to restore service ASAP since because that's the priority.
If this sounds unfamiliar or onerous then it's because you and others might have never experienced teams that do this. Robinhood is clearly lacking this experience and disaster planning.
> strict regulations to follow, and the responsibility of safely handling money for millions of people. This should be fixed within days even if it means redesigning the entire system
Somehow, in my mind, the responsibility of safely handling money of millions and redesigning the entire system don't really mix. Ditto for strict regulations and redesigning the entire system.
I know it's not the same regulations but I'm pretty sure the next FDA auditor would have a heart attack if we slipped "rewrote the whole product" into one of our patch notes at work.
Being in a regulated environment with lots of peoples money should make it harder and take longer to redesign and implement anything. I don’t see a complete redesign of a non-trivial part of a trading platform happening in a week.
Your literal quote was "This should be fixed within days even if it means redesigning the entire system". And my position is, being a regulated financial service, it's probably not reasonable to redesign (and build, test, deploy etc) a system in days.
If they KNEW it was broken and were working on a fix, then they should of shut down the service. I'm sorry but this is simply unacceptable. I don't use Robinhood, but when you are handling very time sensitive contracts that can make or lose people a lot of money in a very short period of time if the service is not stable it should not of been in operation until it was fixed. I can't believe HN is making excuses for this.
If they do that, then they're definitely harming any customers who have a position they need to get out of, as opposed to only potentially harming them.
If you believe that the platform is so broken that it shouldn't be used (and I'm not arguing against that, btw) then a more reasonable solution would be to only allow position-reducing trades. That would also reduce overall trading volume on their platform.
Not that I expect them to do any of this, just pointing out that 'shut everything down' is not necessarily the best approach.
Provision the resources and take the financial hit in costs.
If your architecture wasn't able to scale horizontally because it was poorly designed, heads should roll. This isn't some social network - these are financial platforms where literally the individual customer is financially dependent.
Heads. Should. Roll. Tell us the post-mortems, and then tell us who got the boot. Completely unacceptable.
The only company in the world who can claim to have made a perfectly horizontal system at all levels of macro and micro is amazon, and even they would be lying to do so.
A system like robinhood can have hundreds of moving parts, if 99 of them are horizontally scalable but 1 is not, eventually that 1 piece will become the bottle neck, and the fact it hadn't been made horizontally scalable yet is more likely to be a testament to how much work would have to go into doing so.
> Heads. Should. Roll. Tell us the post-mortems, and then tell us who got the boot. Completely unacceptable.
This is an unfortunate viewpoint. How quickly did we forget that robinhood is literally providing a service that no other company was able to do before. You want 100% uptime, you won't find it in an online service, let alone an online fee free service, you'll find it on the floor of the exchange.
All brokers were able to do it, they just didn't because commissions were easy profit and there was no competition.
$0 commissions aren't a big deal either. If you just buy and hold, it makes no difference. If you trade actively then a real broker with better tech and order management is worth way more than the fees.
> The only company in the world who can claim to have made a perfectly horizontal system at all levels of macro and micro is amazon, and even they would be lying to do so.
Haha, true. Call up you AWS account representative and you may find that certain service limits can’t be increased for love or money.
If you horizontally scale all the way, you just take on different issues like CAP theorem trade-offs. Financial book-keeping is a hard thing to scale and all it takes is a single component that cannot, like the transactional bedrock of your system.
I don't think getting rid of the people who know the system best will help anything. Some new guy who comes in with no clue of what happened or any of the codebase is not going to do a better job. It's just going to reduce long term productivity and increase risk.
I get the impression that the people running Robinhood actually don't understand how difficult it is to implement what they want to offer, both on the reliability side (outages) and on the logical side (infinite margin).
This is pretty interesting for capacity planning. I worked at a minor ecommerce company when Black Friday was still a big deal. In the month leading up to it, we went into code freeze and made sure all our systems can handle added load.
What Robinhood is facing is like that, but with no notice. It means they ~always need to be ready for a 4x(?) increase in load.
This is one of the problems that "the cloud" is supposed to solve. No need to keep infra up for that unicorn event, just to ensure that you can scale properly when it does happen.
Until you hit some kind of aggregation server that it's not trivial to scale out properly unless you take a completely different approach or that you may scale vertically up (but eventually you will hit the ceiling).
What sets financial services apart when they fail is the extreme legibility of the losses. One can argue that if say Reddit is down, they have losses, but measuring the impact in terms of dollars lost is difficult.
It's a lot easier with something like Robinhood. So, ethics aside, the level of measurable risk they have now is existential. Lawsuits will flow.
The market makers and exchanges stop executing orders because a series of circuit-breakers fired (set by the regulating organizations).
Robinhood and other brokers don't execute any trades, they just forward orders from users to their exchange partners and show up to date quotes. They have a much easier load to bear than actual exchanges.
Matt Levine had some interesting (and funny!) insights the last time [0]
> It is well known that one of the best services a retail broker can provide is not answering the phones during a crash. The market is down, the customers panic, their timing is terrible, they want to sell at the bottom, they call you up to say “sell everything,” you say “we’re sorry all our representatives are assisting other customers, your call is important to us,” they hang up and get distracted, the market rallies, they forget about selling, you have saved them a fortune, good work.
Buying the dip always feels intelligent in a bull market. In a bear market, it's not so wise.
The future is unknown to all of us. In some cases, looks are not deceiving. When the market appears to be on the cusp of freefall, it can in fact accelerate downward.
Yes, nobody has a crystal ball. It always feels like other people do though.
It's good to make decisions with a sober view of risk and reward. I feel so much market advice that filters down to us laypeople is whether we should or should not buy or sell, when really it is all based on our level of risk tolerance, how much in terms of assets we have, what our time horizons are, what our expertise is. I would say a healthy young person with plenty of cash to burn might well look at buying stocks now. But a person nearing retirement, perhaps with health issues, they should probably board up and get ready for the storm. Most people below retirement age should probably just stop freaking out and keep contributing to their retirement plan as they (hopefully) have been doing, and make sure to have some cash on hand.
Our society does not do well with self-control or with gray areas. We want easy answers, which, unfortunately, don't usually exist.
This is where the panic goes to though: they might have 2-3 months cash right now, but might feel like they need 12-24 months cash to ride things out. People start worrying they'll lose their jobs and then their runway is at the mercy of the market. If a significant chunk of your wealth is in your house, that isn't easy to get out of during a downturn. Your tolerance for risk goes even further down if you have a family. Even further if your spouse doesn't work.
These are rational decisions that look like panic selling.
Tech work is a field that likely won't have to worry about this though.
>Tech work is a field that likely won't have to worry about this though.
With tech stocks tanking and economic growth likely sluggish for at least the next eighteen months, I can see large tech firms putting a hiring freeze in place, making it more difficult to switch jobs and more difficult to get hired if you DO get laid off.
Moreover, all that RSU comp that makes such a big portion of many tech workers take home pay just took a 20% haircut from a few weeks ago. It could well go lower.
Then there is the startup scene. I can definitely see current events massively impacting funding (see the recent Sequoia post). This could quite reasonably cause many startups to fail just like they did in 2008.
The IPO market is also going to be hit. I don't see any major unicorns doing an offering until the recovery happens. A lack of IPOs means, again, that the people willing to contribute to earlier stages of the pipeline, dry up.
Then there are secondary and tertiary effects. If the average tech worker sees their total comp drop by 10% to 20%, and quite a few lose their jobs, do you think the home you just bought in SF is going to be worth more in a year, or will there be some reduction (in appreciation if not absolute values)?
You are making a fundamental mistake with your assumption that there is a clear distinction between laypeople and experts in the stock-market, there isn't. Money managers, mutual fund managers and hedge funds managers make terrible mistakes and lose money all the time, and are usually the 1st to panic. So, there is no such thing as an "expert" in the stock market.
I could be wrong, but I'd see survivor bias as "at the end of the day, everyone still in the game reports their results", not acknowledging that those doing really poorly might walk away from the game.
Indeed, if anyone things the Virus panic is as bad as it's going to get they're going to get some pretty rude shocks over the next few months.
Right now it really isn't that bad. I'm in London and going to work normally, the city is busy and the trains are still full in the mornings. Our company is starting to migrate people to working from home though. So a proportion of each team is wfh and staying there, for 14 days or until otherwise notified, and more will probably follow not because there is immediate danger, but because we want to be well prepared when there is.
If the markets are like this when there are a few hundred cases in the UK and US, what will they be like when there are tens, or hundreds of thousands of cases and major cities are in lockdown?
Back to investing, stocks are down significantly and it's quite possible now is a good time to start buying, for those with the capital. Maybe spreading out a balanced equity investment portfolio over the next several months. I don't know if or when the market might go lower, or by how much, but it seems unlikely it's going to rally all that much given that we can be pretty sure there's plenty more bad news still to come. So investing too much now might miss further falls, but IMHO buying over several months is unlikely to miss the dip. That's how I see it anyway.
> If the markets are like this when there are a few hundred cases in the UK and US, what will they be like when there are tens, or hundreds of thousands of cases and major cities are in lockdown?
Investors are trying to judge the likelihood of these very events and selling / buying accordingly. Markets are based on predictions.
Exactly. When the virus is at its worst is when the market will start to bounce back because recovery will be priced in. The markets are all about the value of the future. That’s why it’s all kind of a joke to me. Its based on hopes and dreams(or fears and nightmares) instead of how a company is actually doing.
Of course, but those predictions are only based on available information. We probably won’t know which companies will be worst affected, or where. There’s no way anyone could have predicted Lombardy would be the first part of Europe to be hit so hard. When particular companies, regions and industries turn out to be more severely affected, they will take the hit asymmetrically, but that will hurt the overall market. The current impact us being gauged fairly tentatively IMHO.
For example it’s likely several more airlines will go to the wall, but we don’t know which ones yet.
> Right now it really isn't that bad. I'm in London and going to work normally, the city is busy and the trains are still full in the mornings.
I flew from Seattle to San Francisco and back this weekend. In both cases, my "prime time" flights were less than 1/3 full. Returning yesterday, there was literally zero wait at bag check and security, I walked straight up to an agent. Haven't seen an airport like that in years, let alone a major.
> If the markets are like this when there are a few hundred cases in the UK and US, what will they be like when there are tens, or hundreds of thousands of cases and major cities are in lockdown?
Some conservative predictions are arriving at millions of cases in a couple of months. There are too many unknown variables for an accurate prediction but, without really drastic measures, it's difficult to argue with exponential growth.
At some point it will plateau and become another seasonal flu. And we may even get vaccines. But I agree that things are going to become way worse before they improve. Most worrying is the strain on the health care system, which may by itself worse prognosis on other, otherwise unrelated, diseases.
There's also the saying though of "buy when there's blood in the streets, especially if it's your blood". All got to go up sometime. Just how much capital do you have to wait out the drop?
That was coined for investment capital that comes with a cost. E.g. (in its simplest) borrowing money to trade with, and then having to make regular payments to maintain your positions.
Assuming you have no pressing needs for your invested capital (e.g. most retail investors), the market cannot actually stay irrational longer than you can remain solvent.
Time-based options have a fixed cost which you pay up front. So again, there is no risk of insolvency because you've already paid up front and capped your losses.
I view it as such: do you believe we have achieved peak efficiency with regards to production of goods and providing services? If not, there’s room for the economy to grow.
I am not an economist though, this is just my layman’s view.
This isn't quite true. I get what you're saying, but from a technical perspective, it's not accurate. The markets also go up based on the total money supply in the system. If the government takes on a lot of debt, eventually a portion of that makes its way to the stock market, among other places.
You don't actually need to improve efficiency/productivity if the government has tools like the discount rate, open market operations, the ability to alter margin requirements on derivatives, fed funds rate, as well as programs and policies like MMIF, TAF, CPPF, ABCP, TALF, ZIRP, to manipulate the money supply and the velocity/flow rates of money.
The actual question is whether the market thinks that future efficiency gains are on the way or not. If the market does, then that efficiency is already priced in and you can't make any money off of it.
What you actually have to figure out is: is there any unexpected future efficiency gains that the market doesn't expect. You have to know something that the market does not.
> The actual question is whether the market thinks that future efficiency gains are on the way or not. If the market does, then that efficiency is already priced in and you can't make any money off of it.
Isn't this ignoring opportunity cost, uncertainty, etc.? If the expectation of future gains is priced in, wouldn't treasury bonds sell for face value + remaining interest on the secondary market?
We're not near any fundamental limits on growth. They exist, but the economy can get a lot bigger before it hits them. I have heard some really daft arguments about it ("the observable universe is finite!") and some less daft arguments ("resource shortages are more likely than asteroid mining within our lifetimes") but all of these predictions are for things a long time in the future.
Yes, it does. All of the processes that have driven growth in the past are still working, and they will until they reach their limits. That's what it means for them to have limits. Why would people stop engaging in economic activity while it's still possible to do so?
That's a slogan of pro traders who have three problems:
(1) They trade on margin, so huge drops are much more dangerous to them
(2) Their boss sees a Profit & Loss for the trading day that evening, if they don't like what they see they might give their portfolio to somebody else.
(3) Customers are calling the sales desk and they want to sell; hopefully sales can slow them down, but you may need to sell to pay for redemptions. Many customers may need to sell to rebalance their portfolios, get liquidity, etc. This is a mechanism which can carry instability from one market into another one which would otherwise be doing just fine.
When my wife and I bought our first house in 2006, the prevailing wisdom was that real estate was an incredibly safe (if often conservative) investment. Then 2008 happened. From that experience, I've decided that I will never invest in a market I don't understand.
That's the beauty of long-haul investing, though. Two years is nothing in terms of an investment strategy, and real estate, along with pretty much everything else, has rebounded since then.
There are a handful of hyper-geniuses that can make gobs of money by actively trading. For the rest of us, there's buy and forget.
We bought our house in 2009. It was kinda scary--there was no telling when the recovery would come, and I was working a temp job that ended the day we signed papers. But we had been saving for a downpayment for several years--by 2005 or so it was clear that the Great Bubble was going to end very badly, so we just hunkered down in our crummy tenement apartment and waited it out.
The lucky bastards are the ones selling get rich quick schemes. The hyper-geniuses are the ones keeping their mouth shut and managing the Harvard Endowment.
You mean they are genius in the sense that they get paid to manage other people's money?
Because the Harvard endowment, like most all other funds out there, generally fails to beat the market and has had down years even while the market was up. This is probably due to lower risk tolerance, but it doesn't really show that hyper geniuses are running it either.
Recovered in what way? I know a couple that just recently had the price of their home return to the price they paid for it over 10 years ago. With inflation taken into account, the house is still worth less than what the couple got it for.
It obviously depends on where you are (hence _most_ western markets) but in most parts of the UK and Ireland (and indeed many places in the US) if you bought at the _peak_ of 2007, mortgaged to the hilt, if you sold now, you would break even on inflation.
You're also forgetting that they had somewhere to live for a decade for the cost of inflation of the house, which is significantly cheaper than normal rental prices in many places.
To come out even after a decade, after housing costs, seems a pretty "safe" investment to me. Certainly not as lucrative as having invested in an index tracker, but it beats the 0% interest my bank pays on my current account balance.
I remember the drugs company Elan[1] here in Ireland. It got hit bad in the early noughties by accounting issues and other problems, but they had this potentially killer drug called Tysabri and so, as they tanked, there were "smart" investors at every drop buying in and it just kept dropping and kept dropping. Loads of fairly smart people lost a lot of money. Only those who could really follow it all the way to the bottom were ok.
The high was $74 per share. The low was about $2 a share.
On the East coast too, it's not that common. I once heard it by an overly pedantic gentleman... But his pronunciation made me giggle so I started using it.
Maybe it is irrational buyers who are getting the benefit this time. A lot of unknowns right now. Past performance does not always predict future gains.
Seemingly this is the strategy of many Bitcoin exchanges as well, as every time Bitcoin price is going down, mysteriously Coinbase is having issues with their systems.
I just happened to have watched The Big Short [0] yesterday.
In the movie (based on real events) the same thing happened: during the housing crash suddenly every bank had 'technical issues' when people tried to sell their stocks.
Indeed ! If I remember correctly though, they had difficulties only until they sold all their crap (hence lying to their clients by selling them crap as gold). Once they were freed from their toxic assets, they "allowed" everyone to sell. What a world we live in.
Disclaimer: I'm not in finance, don't know much about it, but it's what I understood while watching the movie ;)
That's somewhat correct. But it wasn't that they didn't allow those people to sell. In the film they were dealing with specialty derivative products that weren't traded by your average person. The derivative products in the film were sort of like bond pricing where price discovery isn't obvious and determining a fair price is a bit subjective since the volume was very low. So when people like Michael Burry (Bale's character) called to see what kind of price they could get, the banks would give him a bullshit, obnoxiously low offer price they'd be willing to give him because correctly valuing these derivatives would bankrupt the banks. So they held them on the books with fake valuations to stay afloat while they reduced their risk. Then, after doing all that they could fairly price them, which is when Burry makes a metric shit ton of money basically overnight - because he is finally able to get a fair price.
For example, if we took a simpler example of something more people are familiar with like options...let's say you buy a put at a strike of 100. The stock is trading right around 100. The market tanks like it has been doing and the stock goes down to 80. That's great for you since you had the put. You technically have the right to buy the shares at the market and shove them onto someone and force them to pay you 100. Most of the time, since we're dealing with derivatives, they just trade the value of the option itself and skip dealing with the underlying security because it takes a lot of capital (100 shares * 80 bucks each in this case). So the value of the option (the derivative) should be around $20 as you approach expiration and the premium fades off. If the market maker for the options has way too much exposure and wrote naked puts (meaning they didn't have the underlying security) they took on a ton of risk and basically got fucked. In this scenario it's like Burry calling and finding out his options are worth like $1.50 instead of $20 simply because they are thinly traded and therefore can't get a good price by anyone. This actually happens on some options - you'll see a huge spread between the bid and ask on some options so it's not like it doesn't happen even for this type of retail-friendly derivative.
That's the point of cryptocurrency though: a financial world unencumbered by those annoying regulations that protect the buyer. No takebacks.
Reality is the regulations on the financial system are a feature. Being able to have charges reversed if I get my card skimmed and someone makes a bunch of charges is a feature. Being able to get my money back if the seller ships me broken goods (or nothing at all) is a feature.
Having regulations on banks front-running their customer's transactions is a feature (one that Robinhood unfortunately gets around somehow). Having regulation on securities not being scams is a feature.
> Reality is the regulations on the financial system are a feature.
As with anything, the answer is sometimes.
You want those rules for your retirement account, because if someone can steal a six figure sum from you instantly with no takebacks, that is bad.
But the overhead of that system is not always required, and it's the major reason that we don't have e.g. micropayments. The overhead of allowing transactions to be reversed eats the entire transaction amount for very small transactions.
So it would be good to have a system where you could, for example, transfer $50 into it from your bank account, wait the month or so until the transaction can no longer be reversed, and then transfer it from there a few pennies at a time with insignificant transaction costs because the small transactions are instantaneous, anonymous and can't be reversed. Which isn't nearly as problematic because even if you get scammed, your loss is limited to the $50 you put in. Instead of getting wiped out, you pay a reasonable cost for a personal lesson on security and trust.
And there is no reason we couldn't have both. Then you keep the bulk of your wealth in the inefficient safe system and some petty cash in the one with lower transaction costs and get the best of both worlds. But the existing rules don't allow that, which is bad.
This is a lot different though. Banks were the actual dealers of the securities in the big short- you were actually buying and selling to the bank itself. Robinhood and stock brokers, you aren't buying and selling to them- their primary function is to just give you access to the markets where market makers are participating on the other side of the deal. Robinhood does not participate in market making, they purely give you access to them.
In the Big Short case, the banks were acting as market makers, or more technically in this case dealers- this was more akin to going to a used car lot. You read a lot of reports saying that the new Fords were flying off the lots for outrageous prices, and to people with dubious ability to pay for the sticker prices they did because they got loans they shouldn't have. You go to the dealer and say I want to sell these cars short and buy them back later. The dealer presumably thinks you are an idiot, or wants to hedge some risk (the analogy is breaking down here), but says ok sure, thinking he is going to sell them for even more in 6 months. 6 months later comes- you were right- there is a flood of Fords on the market at half the price because so many people had them repossessed or are desperately trying to get out of these loans that are killing them. You go to the dealer and say "Ok bud, I'd like to buy these cars back at half price..." and they say "Nah, these are still worth 98% of what you sold for... how about that price?" and you are kind of stuck. You bought specific used cars that are kind of but not really entirely fungible. You can't just go down the street and buy those cars and replace them. You have to hope that they feel the pressure to lower those prices because they start feeling the squeeze for cash, or a regulatory agency comes in and puts the pressure on. They can kind of live in la-la land and avoid the reality of the situation as long as they have no requirement to sell.
Eventually though, they blinked and once one bank started taking write downs, all banks did, and once that became acceptable, they all followed suit- and they also needed the cash at that point.
Anyway- now back to stocks/options- these have well known discoverable prices and you can sell them on open markets where the brokers you connect to might be on the other end of the deal but its really unlikely. A broker/dealer like Robinhood or Schwab or Interactive Brokers should theoretically have an entirely flat position at the end of the day.
TL,DR: Robinhood didn't go down because it doesn't want you to sell your stocks.
Same here. I don't remember the details, but I had BTC (transferred from local wallet) stuck in some unexplained "pending" state for more than a week and Coinbase support was nowhere to be found. Thankfully, it was eventually resolved and that was the end of my BTC adventure.
One of the problems with Bitcoin is that there is a finite and fairly low limit of transactions that can take place per block. You have to get the bitcoin onto an exchange before you can trade them (and it is not recommended that you leave them there due to risk of scams/hacking) and during a time of crisis everybody else is trying to move their coins to the exchange too.
This means that transaction fees spike to 10-100x their normal rate during those crises, or else your transaction just sits and takes a week to get confirmed (if it ever does at all - eventually it will just time out after a few weeks).
There's logic to this... Most exchanges have most of their funds offline, and only a small portion on their servers (to reduce losses in case of a hack).
When volumes go up, the servers run out of money, and everything grinds to a halt till humans intervene.
The humans typically want to do a lot of checks they aren't being tricked into refilling the online wallet too. Those checks are usually manual and take time.
The reason is quite simple: they are using Ruby on Rails internally for many parts of the system.
As much as I love Rails, it's extremely hard to scale an application written in it to handle millions of users. All the Active Record transactions have to be rewritten as database transactions if you want to make sure that the databas doesn't lock the tables for a long time.
> The reason is quite simple: they are using Ruby on Rails internally for many parts of the system.
The choice of a language or framework is only one factor on the success of failure of a system.
You can write a broker in brainfuck if you are so inclined. You can make it rock solid. Your pick may require more or less engineering resources, but that's all it does.
You mention ActiveRecord. That too is premature optimization. Crafting optimized SQL queries will not save a badly architected app. They might even be worse if whoever is optimizing them doesn't fully understand what they are doing.
> if you want to make sure that the databas doesn't lock the tables for a long time.
You didn't mention which database you are using. You didn't say anything about its schema. This is yet another generalization. Not all databases will "lock the tables", no matter how badly queries are written. And, if they do, optimize that.
Usually, only a handful of services are critical. Microservices are all the rage, optimize those to death.
Both Twitter and Groupon scaled to many millions of users using RoR. I don't remember it being especially more difficult than scaling any other kind of system to millions of users.
That's not insight, that's malpractice. I mean, maybe the customers are wrong. But the broker's desire not to execute sell orders and feed a panic is absolutely self-interest, and if they're wrong (and they often are! We had a similar panic just last week that turned out not to have been "the bottom"!) then they've deliberately hurt their customers to try to protect their position in the broader industry.
And that's bad. And it's the kind of behavior we tend to see at these overheated moments where the market has gotten away from the fundamentals and all the players are trying to find "tricks" to keep the gravy train rolling.
I don't know a single broker or sales-trader who would intentionally reject orders from a well-meaning customer. Brokerages make money on commissions and order flow when customers trade. The author of this Bloomberg opinion article has a kind of wisenheimer sense of humor and is playing fast and loose with his comments, in an attempt to make a point about the nature of panic.
Failure in moments of market stress is the worst way for a brokerage to protect its position in the broader industry, because downtime damages customer confidence (long-term income) in addition to reducing order flow (short-term income).
Order flow is more valuable in moments of extreme volatility because spreads widen when the market goes nuts.
>I don't know a single broker or sales-trader who would intentionally reject orders from a well-meaning customer.
It happened aplenty in 2008, when banks were rushing to offload toxic crap that they owned on to the market FIRST, and only THEN were they willing to help their customers do the same.
This was true of stocks, but if you are holding options or other derivatives which are time-dependent then a market being closed is worse case scenario. RH is largely known for their options trading after all.
Not being able to exit these high-risk positions won't just bring your account to zero, but potentially put you in the six-figure negative if you are unlucky.
Business Adventures has a similar anecdote, referring to the 1962 crash:
> "Other firms were less fortunate, and in a number of them confusion gained the upper hand so thoroughly that some brokers, tired of trying in vain to get the latest quotations on stocks or to reach their partners on the Exchange floor, are said to have simply thrown up their hands and gone out for a drink. Such unprofessional behavior may have saved their customers a great deal of money."
"Oh, hey, we saw your message but we were just too overwhelmed to execute it. We did however, manage to do the next best thing, which was to purchase an option, expiring today, to sell at the prevailing price at the time of your call, and, to make things right, we'll go ahead and exercise that option, unless you'd prefer we just debit you the cost of the option and let it expire without exercise?"
Honestly, I think this is BS that a broker/platform can just cut off investors from making buy/sell decisions when it’s inconvenient (unprofitable) to handle.
Brokerages aren’t doing investors any favors that are purposefully trying to buy a dip or sell for cash, they are only doing themselves favors as the middle man.
Lame arguments from brokerages don’t change that. Has there been a recent case of “unexpected” downtime during a huge market rally?
You should not assume that you can reasonably control your assets in moments of volatility. You may not be able to control them at all.
Market access is a major component of liquidity. If your broker already holds a phone in either hand, he won't be able to pick up your line when the light next to your name flashes.
Imagine that you have all of your liquid net worth in GLD and a huge cholera outbreak hits lower Manhattan. Bad shellfish. GLD rallies and you want to close your position and take profits. The problem is, the largest firms that make markets electronically on GLD have sent their employees home and shut down for the week. Their absence from the market makes it difficult for you to get a fair price for your large position.
Imagine that you use Verizon for your mobile phone and internet service. You think their service is great, so you buy shares of VZ. One day, Verizon announces that due to a malicious hacker, their network is experiencing cascading nationwide hardware failures that will not be repaired for at least a week. VZ stock plunges, but you can't modify your position because you can't get on the internet or make calls.
A less fanciful example would be owning stock when trading has been halted by the exchange. Perhaps there exists a buyer during the halt who would be willing to trade at a price that is favorable to you, but that transaction cannot happen.
Tail risk, in some cases, is a scenario where stocks get cheap. In other cases, stocks can't trade at all. (And in the worst of situations, tail risk is a scenario where guns get expensive.)
I'd consider all of those example cases where it's not necessarily reasonable to control your assets. I will say the Verizon case is a bit fanciful because it should be pretty easy to hop on another network quickly for most people to bypass that small hurdle...
When we're talking about an enterprise scale electronic trading system that should already have tested these sort of load scenarios, especially in a world where AWS/Azure/etc. exist and elastic computing is the target roll out for applications that probably don't even need scaling, that's not a reasonable case. That's a case where something that should already have been planned for and tested thoroughly slipped by. It's especially damaging when it happens repeatedly over a short time span. There's nothing unpredictable here. Management and IT at Robinhood surely read the news and knew there would be massive load today.
Surely, someone designing systems and software in this domain understand that the market could be highly volatile and their system needs to be ready for those cases where it's flooded by customer requests or clearly conveyed to customers that it's not ready for such cases.
Robinhood operate on razor-thin margins and if you just look at where three-month implied vols were trading in early Jan, we are in a 7-sigma event. To be fair to Robinhood, it is probably hard to anticipate and prepare for such an event when you're not charging commissions. They are not the only brokerage that has had problems in the past seven days. And they lose money when their customers can't place orders.
"You should assume you can reasonably control your assets at any given time." Sometimes you can't reasonably control your assets, because you can't control them at all. So you shouldn't assume that you can reasonably control them in those times.
I hope it is clear that I am not trying to be argumentative; I actually think it's a complete let-down that Robinhood has left so many people frustrated as their wealth whip-saws around in the market. I'm just saying that tail scenarios don't usually fit into our predictive model of what's "reasonable," even when the solutions seem crystal clear in retrospect.
On a more general point, I never understood why market returns are assumed to be gaussian - these "one in a billion" events seem to happen far too often.
Their target audience is casual investors. The kind of people easily influenced by reddit/twitter sentiment.
The fact that wallstreetbets made a PREEMPTIVE RH goes down thread [0] (2500 comments) is pretty conclusive proof that they're never living this one down.
This is a good reminder that all these new mobile(brokers|banks|&c.) are not as good as the historical systems that have been stress tested for decades. All is good when everything works as expected but as soon as the slightest issue arise you're in for a ride.
That's what I think of every time I see a post about some new VC backed startup that says they are going to disrupt an industry by creating a lean product that cuts out all of the unnecessary cruft and legacy systems. People need to learn that there's usually a reason something is done a certain way, and having barrels of VC money doesn't make you an expert in a field.
Sometimes the reason it's done that way, is because they were in a rush in the first iteration, and never got around to refactoring.
Loads of VC money means you can take more time up front to architect things better. Maybe have more experiences engineers and architects on the team from the start.
Not really. VCs expect 10x returns. Yesterday. They really don't care if it's the best system ever designed. It just has to work until a liquidity event.
I understand your point, but I think VC backed companies are likely in a more hyper growth mindset than banks and conservative industries were when they were making these systems in the 80’s.
As much flak as I’ll get for saying this, why Python? It seems like Robinhood went all in on an ecosystem that is easy to get an MVP off of the ground with, but certainly not as fast and reliable as you would expect a system moving billions everyday. Language typically doesn’t matter for end users, but man, there are at least 10 solid options to build a critical trading system in that are faster and safer than Python.
Is it really 100% python? I can't find any indication of what they use besides the python helper library for Kafka. Seems like a message queue would be conducive to partial rewriting of the system.
Pasting this has recently become a meme. Not sure it applies here, as it's not about removing stuff from the same instance, but about a competitive lean product?
It's about removing legacy systems and pretty directly about reforming xyz industry with the new tech, so I think the principle of reform requiring knowing why something you want to get rid be of is there in the first place is directly related to the topic
A quirk of information propagation. If it's there, it was put there for a reason. If you don't understand the reason, you're at risk of having to rediscover it the hard way; something we generally wish to avoid at all costs.
Which is somewhat ironic considering the attitude around things like Trade Secrets leading to a plethora of Chesterton's Fences being basically guaranteed to litter the economic landscape.
I think the real irony is that much of the time fences exist because of the quirks (fences) of some other system you are stuck with. Sometimes it feels like it is fences all the way down. Other times, you work your way down 5 layers of dependencies and discover that the whole pile of ugliness is because the core system is designed around physically mailing boxes full of reel-to-reel tapes. When that is the case, there is potential for a cascade of "lean modern apps" to de-cruft an industry.
Ultimately, RH is backed by a legacy system somewhere. I believe they custom built their OMS (order management system) which was the previous bottleneck but somewhere between RH and the other side of the order is a legacy system that they can't disrupt.
This might be slightly unfair to Robinhood ... the time we saw volatility similar to this was in 2008. Most of the major banks had technical difficulties that day (despite being around for decades of years).
Technology and scaling are tough. Reddit, Twitter and CloudFlare have gone down in the last 12 months, despite being constantly stress-tested.
My direct trading account at my (very) old bank is also saying it's having technical difficulties today. It sounds like display issues, not trade execution issues though.
I know Fidelity changed their default dashboards to not load data-heavy views for the past week or two. Usually you get a nice dashboard page with some charts and personalized info, but right now you get a few generic (non-customized) market stats and a bunch of generic links.
Still useful enough, and if you want, you can get to your own dashboard... but things like this (inconsequential as they seem) can be the difference between going down or not.
Each non-personalized element on the page that's removed saves at least one DB query (and in a fancy app like Robinhood's case, maybe a half dozen).
It's always been either completely down or having a huge lag in executing orders at market open for the last 6 business days. That leap year bug was a hoax. Robinhood just can't scale under heavy load.
I wonder if people will leave the platform en masse to alternatives as most brokers have no fees on trades these days.
The company never said their downtime was related to the leap year afaik, that was some random twitter thread looking at requests on their browser that had datestamps from the future (which could have other explanations). Unless they're rolling their own date library (which is very unlikely), the code should be using the system time of whatever computer the software is running on, meaning it would be extremely unlikely that the leap year caused a bug.
You are correct in that it's more likely their system just can't scale.
I did just that early last year when I realized that they simply don’t have the infrastructure, support personnel or sense of urgency I needed from a broker to handle my money. This is pre-free trades — I decided that I rather pay a fee than roll the dice on when they’ll clear trades and bank transfers (in one instance an ACH took 2-weeks, with canned messages from support).
I don't see how it's worse to do it on a phone than your laptop, or over the phone with a broker via a voice call?
Most folks I know who use RH just use it as a fun little side gamble/game. Small-ish amounts of money, something they could afford to lose, with the rest in more stable funds or investments.
I agree, both that that's my experience and that it's the right way to use RH. But then that means this isn't exactly an existential crisis; if you can't run your side gamble during some periods of high market volatility, does it really matter?
On one hand, now is a fun/important time to be messing with your side gamble. It'd be like if a betting site went down the day of the Super Bowl. If you're just using RH to gamble and day-trade, periods of high volatility are some of the most fun times to be actively trading!
On the other hand, I don't really care. I'll be fine. Most of the folks I know who use RH are just joking about it being down. It's annoying, but no one's broken up about it. A lot of the folks casting their anger at RH seem to be folks who already dislike it and probably don't use it?
Professional or large-scaled investors with large sums invested probably aren't using RH for a lot of reasons even before you get to the downtime issues.
Merrill Lynch / Bank of America's trading platform is also down... so this isn't limited to Robin Hood. I was completely unable to place any trades or even check balances during the market opening this morning.
However, engineering HEADS SHOULD ROLL. In the era of on demand computing, near limitless cloud resources, etc, these engineering teams should have already had capacity planning ready to go for these trading platforms. I'm extremely disappointed - these engineers LIVE for this stuff, and they dropped the ball.
Again, heads should roll. This is not some data center that caught fire, this is capacity planning at its lowest common denominator.
1) you make it sound like scaling a system, especially one doing lots of financial transactions, is trivial. Just because you can get access to nearly unlimited computing resources doesn’t mean everything just magically scales infinitely.
2) why should heads in engineering roll? Do you know it was incompetence on their part? Do you even know what the issue is? How did you diagnose the fundamental issue here without any further info?
I’d hate to do a post mortem with you. Sounds like you’ll be basically calling people involved idiots and for them to be fired.
he is the customer, not a employee. As a customer I don't actually care about postmortems, i just want things to work.
If it doesn't work, I use another product. I'm leaving robinhood due to their issues, and it sucks it takes so long due to their transfer limits
That's fine and for every customer to evaluate for themselves. But there is a difference between saying "as a customer the product isn't acceptable so I will use something else" and "engineering is incompetent and someone should be fired". Especially if you don't actually know what happened.
The market hit a circuit breaker this morning when it was 7% down. No one could trade at the open, not even professionals, because the market was stopped.
No, I couldn't event place any orders whatsoever, and couldn't check balances etc. Order PLACEMENT and order execution are two different things. The former was impossible as well.
Designing a real time system to scale to this magnitude is not easy at all. I wonder how many engineers would be capable of designing such a system, I sure couldn't. This isn't like most scaling where delaying/queuing can be an acceptable trade off.
Maybe they shouldn’t be in business if they can’t hire the engineering to do the job. Every other discount broker seems to have no problem hiring this level of talent.
Sometimes engineers are not at fault. They are well aware of technical debt and possible performance mine fields. It's the management that does not properly prioritize performance and scale work because there is no apparent "value" to the customer. Until it's too late.
Agree that heads should role, but it may very well be management decisions that brought them to this point. We don't know if engineering was resource/strategically limited (seems unlikely at a well funded startup but it may have been). Having worked places where terminations were the result of failures, it's not always the party most responsible that is let go.
Bad idea. You would be just rolling heads of people who just learnt a valuable lesson in high throughput services. Keep them around, they'll get better than hiring someone else who will have to spend more time learning the same lesson.
There should be accountability for sure. I am not sure if rolling heads is enough or even the right thing by itself - but there definitely needs to be a 3rd party audit, and a public report.
Honestly, I don't. There is enough public information about the brash attitudes of this company's execs that anyone working for a company like this should know that the lowest level engineers working on a technical problem like this are eventually just going to get thrown under the bus.
I hope this is a lesson learned for smart and impressionable engineers about where to work.
The implication from the parent comment is that these individuals are going to go through some emotional strain.
My point - it's difficult for me to feel the same anxiety someone might feel in this situation when they equally ignored the risks of something that is easily identifiable. They very likely felt elated when they joined because the company was seen as cool, hip, and by all objective means is/was successful (valuation, # of users, etc).
In startup land, a junior developer is another startups senior developer, and same applies to senior developer <> VP. Simply down to what the startup you're working at. Might not be that far.
In the big-name Wall Street banks 1/3 of employees are VP or higher.
Senior Developer in the "outside world" typically means Associate in the banks, and VP is the next rung up from Associate, so VP really is just one hop from Senior Developer.
The simple way to get a job like that, is to apply for it. Instead of waiting for someone to retire in your current company for instance.
And its not unheard of. My sister-in-law worked from startup contributor to Cisco VP by applying for the next job up, one at a time, from company to company.
It's just a weird financial thing. Everyone gets to be a VP. I was actually a bit embarrassed by having the title when I worked at BofA. It's not even a manager level. I was what would be a senior developer anywhere else.
Robinhood blamed their DNS system being overloaded last time [1]. Also RH (and other brokers) only show quotes and forward orders with some basic calculations to their exchange partners. They should not be anywhere near the same load as the exchanges. All this points to major architectural problems at the company.
That was a pretty weak explanation, only a single sentence referencing the actual technical issue with some stuff about it causing some knock-on outages elsewhere:
> We now understand the cause of the outage was stress on our infrastructure—which struggled with unprecedented load. That in turn led to a “thundering herd” effect—triggering a failure of our DNS system.
I was planning to wait for the post-mortem because I dislike speculating and I'm not personally invested here, so I prefer to wait. I'm curious when they'll release the full one because it's going to be one of the most interesting ones in a while, given the large scale of the failure and potential implications... (ignoring today's outage)
I'd be curious to know the root cause(s) of their outages. I presume they're using Kafka as an event bus, based on their Faust library[1]:
Faust is a stream processing library, porting the ideas
from Kafka Streams to Python.
It is used at Robinhood to build high performance distributed systems and
real-time data pipelines that process billions of events every day.
I did some evaluation of various Kafka ingestion methods a while back (including Faust), and didn't find Python to be a great fit, so I'd be curious to know if that has anything to do with it.
I had looked at their job board to see their technologies and saw heavy use of python. I know large systems can be built in python, but for financial services something more performant and safe probably would have been better. But who knows, maybe Robinhood would never have had the adoption its had thus far if it chose something else.
Man, I picked a hell of a time to dip my toe into the investment game. I picked up a couple of reasonably wide Vanguard ETFs that seemed to be doing fine at the end of last year. I'm currently down 12%, and down 15% in the last month.
I suppose that's what I get. I realize that I should be looking at yearly data and I've only had them for a couple of months, but it's a little hard to look at.
Not to mention with all of the other RobinHood issues I want to move to another platform I'm already on (I've got an account with Schwab for work stuff and they seem fine), but general wisdom says to hold and let this craziness ride out.
Word of warning though: Buying and holding for a few decades is excellent advice, but be sure to log in every once in awhile lest you get escheated[1] out of your nest egg!
IF: you believe in the long term future of the USA.
THEN: dollar-cost-average into the market. Put a fixed amount to work every month. This has been a terrific strategy over the last 50 years or more.
But, caution, you probably don't want to buy a few shares per month of a stock or an ETF in a taxable account. That's because each transaction is a separate taxable event when you go to sell. It would be a nightmare for you to track this yourself. I have no idea how well RobinHood tracks that for you.
If you are investing in an IRA then no worry. The transactions aren't taxable. Of course an IRA has its own problems. Outside of an IRA long-term capital gains are taxed at a lower rate than ordinary income. But when you withdraw funds from an IRA you are taxed at the higher rate.
A Roth IRA doesn't have those tax disadvantages. I've seen a lot of press lately on this issue. For example, in Oregon, the state itself is promoting Roth IRA investing for the "little people":
Participants saving through OregonSaves beneficially own and have control over their Roth IRAshttps://www.oregonsaves.com
All brokers are now required to record and transfer cost basis information for most assets. It happens pretty seamlessly.
There are also tremendous advantages to having differently priced lots -- you can sell the expensive ones, and donate the cheap ones to charity to minimize taxes.
Yep. I don't have a stomach for it. So I've invested my RRSPs into a few different index funds run by Wealthsimple and Tangerine. I literally haven't looked at them in a year. Short term performance is noise, not signal.
Just start a monthly savings agreement. Presumably you've barely put in any signifncant part of your assets. You're much better off than someone who has $400k in the market and might need it soon.
In fact, starting your investment career during a 15% drop is great to inoculate you against panicking in volatile times.
I just wrote a post about /r/Wallstreetbets reaction to this. I graphed out comments for several different brokers over time, and also mapped how many users were holding SPY puts and got trapped.
At this point, I think Robinhood has to be a dead brokerage walking. I cannot fathom any new potential customers looking at brokerage options today and deciding that Robinhood best suits their needs. Their largest competitive advantage was the free trades, and one can find that at most of their competitors now. Existing customers are also leaving (and possibly suing). The future looks pretty grim for Robinhood.
Robinhood's interface is much better than other financial institutions by far, It's one of the most intuitive interface design unrelated to financial products which makes it very pleasing to use. Robinhood disrupt fintech from every standpoint and we should give them credit for that. It opened the doors and allowed younger generations to invest in the stock market where other players failed terribly to do that. Bad days are happening for them in a terrible timing but I believe they will bounce back, I sure hope they do.
I thought that the other benefit would be easy account setup for options trading? The brokers I'm aware of 'interview' you before giving you the ability to trade options.
I am able to trade options on Merrill Edge after filling out a form. I think perhaps Robinhood allows people to trade smaller contracts than usual? I can only trade options in batches of 100, which means putting a fair bit of money on the line for any given trade.
I work on trading systems. My uptime record was 9 months. Only went down because the DBs went down and our 3rd-party ODBC lib went nuts and hung on reconnect. We traded world wide, so the systems had to be up nearly 24/7. We had a 2 hour maintenance window on the weekends. I know some shops that are US only and start and stop their systems daily.
I took stability quite seriously. My first boss was fired after knocking is out of the options market at open on a Monday. Refused to accept responsibility and opened the firm up to a potential $600K fine. He was out the door before noon.
We've already learned in the last couple years that there are no physical characteristics whatsoever that are associated with gender. And no mental characteristics either.
Does anyone have a good path forward of how Robinhood can even survive after this? Other brokers are not going down, and also offer free trades. I don't even see the point of it anymore, especially if it is going down in some of the most important market moments.
I have a crazy strategy, but hear me out: They do nothing.
The vast majority of Robinhood's customers aren't day trading clamoring to get their daily dose of tendies. The proportion of RH's customers who are actually effected is probably quite minimal, and of those some might change to another broker, but most will probably just put up with it. The market isn't going to be this active for long, and so once it calms down RH will put in place some fixes for its systems. This will be forgotten after a couple of weeks, it'll perform better next time and it'll have almost no impact on RH's value.
FWIW, Charles Schwab is also down, and is often down during heavy trading. There's a big reliability gap between straight consumer brokers, and the next tier up with interactive brokers et al.
yes, this. many other systems go down in explosive volatility, including IB, tda thinkorswim, etc and they all have their ass covered with electronic trading risk agreements but you dont hear anything about that on wsb because theyre all casual RH chunkers
And this is what happens when you live wildly on the brink of tech fantasy and tech brilliance.
I believe this will be Robinhood's undoing and the company will likely just firesell itself (less than $1B) to Schwab/TD so they can acquire the younger demographic.
This many incidents makes FINRA a joke. Time for some outside regulators to step in and require certain levels of availability with real consequences if not maintained.
RH is another one of these trendy "we only hire the best" startups that discriminates against neuro-atypicals with poor social skills who are not good "culture fits"--you know, the kind of people actually detail-oriented enough to make sure leap days are handled correctly.
Why not? I think now would be the most interesting time to be working there, where you can learn the most in the smallest amount of time. You'd gain much more knowledge powering through the hard times rather than just coasting around for the easy ones.
Yeah but what you’re learning is “how to not catastrophically fail your customers”. In a better company that’s a lesson you already learned. There is going to be a LOT of stress and tension, it's hardly a great learning environment.
>In a better company that’s a lesson the company already learned.
FTFY. Just because you work at a company doesn't mean you were around to experience what ever event provided some sort of "learning" experience. If you don't experience it personally, what ever systems the company has in place to protect itself from said events just look like bloat/legacy cruft ripe for a young upstart to disrupt away.
If RH had any interest in being a sustainable business that respects their customers they would have hired engineers that have already been through the "hard times", several times, and don't need to learn on the job. But in startup land sustainable businesses are not the goal - customers be damned.
It's pretty easy to find yourself creating bottlenecked points of failure in systems that expect consistency. There may not be the knobs necessary to scale up past a certain point.
A sufficiently bad positive feedback loop can also drive you into the ditch no matter how monstrous your hardware is.
It requires very good engineering, from software to hardware, from Webml to pricing to OMS to connectivity to trade large order volumes during a storm.
Robin Hood took market share by being cheaper than other brokers. Cheaper rarely means better (more robust, reliable, etc). That's the price you pay for not paying higher fees.
Full disclosure: I shop around for low fees. But that's because I buy and hold boring index funds. It makes no difference to me whether I can trade into or out of a position today or even this week. If you need to change positions quickly or die, for God's sake, don't pick the cheapest supplier.
Did they move fast and growth hack ? There is an absolute culture in the valley now of showing quarterly results and ship something and inevitable trade-off is eng excellence. This culture needs to change dramatically
I'm getting off the platform after this. I had a tiny final trade (wasn't able to sell due to this) and still waiting on my first amount of funds to get through KYC to go to my bank account (I sold on the previous RB downtime).
Because for responsible investors not swing trading the platform works just fine. I buy and accumulate shares over time and haven't had any issues. There might be some outage today but I was able to buy more shares without any issue
Best UI of any platform, very user-friendly, no options trading commissions (although any serious options trader wouldn't care about that). Those are the reasons people continue to use it.
Easy interface, easy signup, and able to trade derivatives without a mountain of paperwork. I don't use it for the family investment company, but I get why people just starting out do.
Explain-Like-I'm-Five: Why would you use Robin Hood instead of a normal brokerage like Vanguard, T. Rowe Price, Fidelity, Charles Schwab, etc that..
(1) already have commission free trades,
(2) an app to make those trades, and
(3) don't charge any fee for transferring your holdings in-kind to another institution.
- When I created my Robinhood account, none of those brokerages had commission free trades. Also, what about commissions and exercise fees associated with stock options? Also free? I doubt it.
- Still today, the UX of all of those other apps/websites (that I've used) is horribly dated, clunky, and slow. Robinhood is amazing to use.
- Robinhood has a Cash Management account that pays me 1.35% interest on all my non-invested cash. None of these other services come close to competing with this.
- Robinhood started a progressive movement with the mission of making stock trading more accessible to the lower/middle class. I love this company and brand. Schwab is just another faceless corporation who only offers free commissions because they had to to stay relevant.
Oh, is it 10:45 am on a Monday already? After the first two days, I downloaded another broker and determined to transfer, but it's kind of a pain. Guess I have to actually do it now.
Not cool enough. In all seriousness, from the features/UI/etc.. Robihood seems to be aimed more at speculative trades as opposed to 'playing the long game'.
The RH app is nicer for simple trades. From open, authentication, to trade can be just a few seconds. Fidelity app feels very heavy opening just to get to a trade prompt.
Not to don the tinfoil hat, but the simplest answer is that Robinhood can't afford to cover the consequences of a fire sale, so they take the app down and blame technical difficulties.
I have no faith that this company is operating in good faith.
In what universe is such a theory of vast conspiracy simpler than their servers being overloaded on a busy day?
When people sell stocks on RH (or anywhere else) it’s not the broker paying the sales rate. So you’re alleging something like RH having fraudulently sold customers’ stock before?
And the market was moving the other way when RH also shut down last week. This would be the only broker in history that had its system set up in a way where they suffer with any trade.
> Robinhood can't afford to cover the consequences of a fire sale
They make more money when trading volume is high. What they can't afford is losing customers to either competitors or customers realizing day trading is a bad idea.
It is not, was just confirmed on CNBC that Robinhood is experiencing system wide issues long after the market reopened...
Frankly, if I am an user of this platform, I'll get my money out first thing after the fix the access...
If the can't keep the system running straight for so many days, who knows what other bugs they have over there...
Robinhood is one of the reasons there's a crash in the first place. People trading stocks on the bus etc.. Heyzeus. None of these people are adding information to the market, they're just piling in with money and randomness.
This is the same frothiness I remember from the .com when ETrade etc. just became big and all these zillions of people who thought they were investing plowed their money in.
The pandemic is bursting the bubble, it's not the underlying cause of the problem. The conditions have been building for several years.
In 2001, the bubble was burst finally by big shift in interest rates from the Fed, and some insane valuations which cause the smart money to run, causing a stampede.
Most analysts have been raising red flags for a long time now. The market has been operating on hype and irrational good vibes for at least a few years.
Warren Buffet started hoarding cash some time ago - long before the Coronavirus.
Listen to Robert Shiller's interview earlier this year on the FT concerning 'narratives'. I don't think it's a new concept, rather just a modern articulation of it. The mania we're seeing is just a more extreme form of narrative.
I've been looking at the Shiller PE Ratio [1] with raised eyebrows as much as the next guy, I'm just not seeing how discount brokerages specifically are at fault.
So far the S&P 500 has lost just one year of gains. I havn't done deep analysis but to me that still seems consistent with expectations that the economy will slow down for months due to sickness and lockdowns all over the world caused by the pandemic (+ this weekend's oil price thing).
The PE ratio of the S&P 500 is still 20, so there is still some way down to go before I would call this the bubble busting .I don't have to hubris to predict how much more the market will fall tomorrow or in 3 months.
https://news.ycombinator.com/item?id=22480260
https://news.ycombinator.com/item?id=22477567
https://news.ycombinator.com/item?id=22475019
https://news.ycombinator.com/item?id=22468361
https://news.ycombinator.com/item?id=22465178