Qasar here (x-YC founder and partner). So im building a simulation company. Not commenting on Uber specifically but the reason lots of companies don't have great efforts is simulation is actually really hard. It’s not building a web application (tested methodologies, established best practices, lots of experienced engineers in the space...etc).
You can get a lot done (and get wonderful demos) through research vehicles. The issue with sim is similar to the issues with AVs as a whole - getting to 90% is much easier than getting to 99.9999%.
Also, there is this weird thing in AV (and specifically in-house sim) where pointy-haired managers think "if we put 30 SWEs on the problem - we'll make progress!". While raw eng horsepower matters in some software problems, when you are on the line between research and engineering, more SWEs might actually make getting a robust product up harder.
Another thing we've seen is in-house efforts overfit solutions. They don't really have a wide viewpoint (nature of how secretive the space is), and therefore go down paths which are just plain wrong but no one is there to challenge them.
Nobody (well, theoretically someone, I'd like to be proven wrong) has a self-driving algorithm which is formally proven. If we can't get, to use your example, web engineers to formally prove their web applications, when they have all these structural advantages, then how can we expect something unproven (har, har) to be formally proven? So there needs to be an extensive QA test suite.
And for all of these other relatively-mature engineering sectors like web engineering, how many of these companies are really investing in QA? In strong and comprehensive test suites, in security reviews, in performance testing and UX scoring before release? Almost nobody, right? I mean, the only reason why society can get civil and biomedical engineers to submit to strict quality control is through regulation, but no regulatory body has any real idea of how to apply that to software. And autonomous driving systems are supposed to be different?
Mythical Man-Month came out 38 years ago. The Pareto Principle, not so much as being explicitly a 80/20 split but about how the vast majority of the effort is needed to achieve actual completion rather than just a proof of concept, was originally coined in 1896. Suddenly it doesn't apply to Uber?
Is it really so surprising that it's so difficult to find senior practioners in these areas when few companies are committed to that level of quality, and the companies which are tend to be so successful that they can afford to pay compensation at rates so far above the rest of the market so as to ensure that few of their senior engineers ever leave?
The reason why there isn't perfect QA for things like webapps is a cost reason, not a knowledge reason.
Society usually doesn't need a formally proven cat pics sharing app, and most web apps, including facebook, are that level of application. They would rather pay a cheaper price than to have a perfect expensive thing.
You'll also notice that as organizations get bigger, they do invest more in testing, QA and regression detection.
A counter example is in the aerospace industry. Most software has to be d-178b certified, which requires not only testing with complete code coverage, but also complete branch coverage. Every possible outcome in every “if” statement must be tested.
Needless to say, this is expensive. I’ve heard estimates that it costs roughly $1000/line of code.
The point is that there are methods to increase reliability, but there isn’t the financial incentive to do so.
Separate point, I’d argue that small organizations also have quite a bit to gain from a solid testing program.
If there is only a single developer working on a project, that developer can manage much much more code if there are solid tests behind it than if she was just winging it with every commit.
At some point the developer starts to forget the code they write and starts relying on the tests they built long ago.
Agreed. AV regulation is early and we're involved in that conversation. We (and I think many in this business) deeply care about safety and believe better technical understanding among lawmakers around software safety will result in better laws. Re: mythical man-month and pareto - 100% true for all AV makers but theory and practice is different. I imagine both their investors and their perception of market forces drive decisions to some degree. This is where you see some gaps (not always) between traditional software lineage companies and traditional OEMs.
I always assumed people would integrate an external simulator like CarMaker rather than build their own.
I'd be exceedingly interested in hearing about your product when it comes available (although my interest is in simulation for maritime autonomy rather than automotive).
We're super familiar with CarMaker -- great for dynamics and some level 2 stuff but there are some big difference between autonomy levels (from simulation perspective at least). At higher levels, autonomy algo development is more like modern software development. CarMarker is a fantastic product (along with CarSim, dSpace ASM...etc) but they were built for a different use cases. A very crude analogy would be tools for J2MEE vs modern tools for iOS/Android. Our product is available, just not very public about our more detailed approaches because that is really where our IP is. Feel free to reach out if you want to chat. Obv, i'm super interested in the topic.
I'm Pakistani and one of two partners at YC who works closest with Pakible. This thought has literally never crossed my mind in 5 months since we funded them.
Something that might not be obvious to those who haven't been is that being in person at startup school is markedly different than just watching the talks online. Mainly that it's one of the few events where there is a critical mass of like minded people who are seriously looking to start companies. Perhaps it's because there is a common objective and a high caliber of participants, I always leave startup school energized and inspired.
I worked at Google for 3 years and GM for 5 years (including engineering in Cadillac) so when I see news about the companies, my ears tend to perk up.
A few things should be set straight. Google was not (and by a long shot) the first to think of and try to develop self driving technology. I'm disappointed to see technical people in silicon valley not think a bit more critically. All OEMs and many tier 1 suppliers have thought about and (especially larger players) tried to build self driving cars. When you're a company like Denso or Bosch, this is a massive opportunity and not to aim a few people of your hundreds of thousands of employees to the problem is hard to imagine even if you know nothing about automotive. To be clear when I worked at Bosch from 2004-06, this tech was in development.
Secondly, the HN crowd specifically is notably biased. Check out Google self driving announcements vs GM (or any news about automotive manufactures other than Tesla). Having worked at both companies in product and engineering - yes they are of course different. But not nearly as much as you think. The single most notable difference is that Google is a very margin rich business. That allows the company to attract and retain top talent but it doesn't mean engineers that doesn't work at Google are bozos.
Thirdly, the technology that traditional OEMs have tried to develop is around 2 axis - cost and safety. Google is approaching the problem differently. Google's approach of a $x0,000 LiDAR system might prove to work if LiDar costs fall but thats a big risk. Also, Google admits that their cars don't work in things like rain. I'm not saying Google won't win this space - the contrary, I think they have a great shot. What i'm saying is this is significantly complex tech and its complexity is hard to understate. There can and will be different solutions that work.
Lastly, GM and Google are not the only ones in the game. Nissan and Mercedes have announced they will have cars with similar tech in 2017/18. Ever single automotive player is investing huge amounts of resources.
While this may be true, it's also been the observation of a lot of people here that, historically, it's been difficult for non-software industries to build effective consumer facing software teams, no matter how well capitalized.
It's not inconceivable that car companies are simply not equipped to manage the kind of complexity intrinsic in building fully autonomous vehicles, independent of the amount of resources they deploy.
I agree that automakers haven't been the greatest with consumer facing software/UI; I've yet to use a stock touchscreen interface in a car that doesn't cause bouts of rage.
However, automakers are probably up there with the best of the best when it comes to embedded "replace the consumer" software. Almost every aspect of a modern car is software controlled or software monitored in some way. I think that the fact that many people don't see their car as a network of computers, or recognize that it's pretty much all software is proof of that. Automakers have slowly been replacing the driver with software, and many people don't notice or think of it like that, because it just works, requires no input or interaction, and doesn't fail.
Honestly I think that in the end, Google will probably end up with a better product, but today, there's probably no one better equipped than car companies to make autonomous vehicles that align with consumer expectations of what an autonomous vehicle should be, both in engineering and resources.
Edited to add:
Look at what's already been done as far as software controlled driving. Mercedes has a system that will stop your car if you're about to rear end someone/hit something. A bunch of manufacturers have systems that will warn you when you're drifting out of your lane. Maintaining speed to the car in front of you instead of just setting a fixed cruise control speed is getting pretty widespread too. All of these by themselves seem like simple iterations, but when you start putting them all in one package, its moving closer and closer to an autonomous car, and all of it works so well that many people don't even notice.
Alexis just told me about these comments. You guys know that I can't impact your careers anymore right! :) Jokes aside, thank you for the kind words. It was an amazing few years.
I'd love to receive some advice on this topic, Qasar, any chance you could forward me your pointers, too? Plenty of specifics I'd like to know, if there's potential for follow-up.
Warning flags should get raised when statements are made about how a company's health makes more sense "when we stop trying to understand it by looking at financial statements".
In finance, creativity is not an asset and the standardization allows the public a structure to critically assess the organization. Specifically against other similar organizations. Of course, if there were financial statements made specifically for (let's say) Twitter - Twitter could highlight all their strengths and quietly hide their weaknesses. I'm not sure that's what is best for the public.
"But the "mystery" of Amazon's surging valuation over recent years becomes a lot less mysterious when we stop trying to understand it by looking at financial statements that were developed long ago for bondholders in an industrial economy, not for stockholders in an information economy – statements that don't tell us what matters most to investors."
The article had nothing to do with creative accounting or misleading financial statements.
To the contrary, Amazon's compliance with accounting rules is a key part of the author's argument -- that Amazon's expenditures on R&D and advertising, though resulting in lower earnings per share right now, are investments that will pay off in the future.
After spending 2 years of my life in a retail turnaround backed by an activist investor (Eddie Lampert at SHLD), I can't say i'm that surprised.
This game of retail is much harder than it looks from afar. Certainly much harder than working at places like Google or Apple - mainly because the market forces are against you. You're working with low margins, high capex, deteriorating infrastructure, deflated brands, legacy technology and a legacy customer base. To top it off, the shear momentum of companies this large (at SHLD we had 300,000 employees) is alone hard to grapple with even if you didn't have market forces against you. It's a daunting task.
If non-linear equity growth was easy, everyone would be doing it. Turnarounds are not for the weak of heart. Turnarounds in that sense, are actually similar to startups.
Was Eddie Lampert ever serious about turning it around though, from a retail perspective? From what I recall his actions showed that he simply wanted to bleed Sears and KMart dry. He slashed capex and refused to make any tangible improvements to the retail experience, his whole focus seemingly on liquidating assets and reallocating capital away from the stores.
I disagree. I think working at a large company for long periods of time can actually stifle your ability to be a great founder. You learn bad habits and your intuition can be deformed by things which help you become successful at Big Co but are useless in early stage companies.
Also, I'm at Google and I know John and his work really well. It's almost laughable to imply it was involuntary. John is an all star. The guy actually wants to do a startup because he find the challenge interesting.
What's fascinating for a company like Google in SV is no matter how good things are, you have to constantly fight to keep folks like John from trying their hand at early stage life. Overall I think it's great for the ecosystem. This type of non-traditional churn (where employers not only have to be the best of the "Big Co.s" but better than doing something completely on your) own pushes everyone to create environments that are not only innovative and rewarding but also fulfilling.
Being a founder and now at Google (John and I started almost the same time and I know him well), I think what's notable about this post is when people usually write "Why I left Google" they are sometimes backhanded compliments or thinly veiled complaints.
What John is saying "I had a great time and learned a lot but I still think starting a company is more interesting".
Worked with John briefly when he was with analytics. Great energy and I wish him the best (bummed he is leaving, though)! He's dead on about how much Google invests in their employees. The value of the things I have learned and am still continuing to learn here have been nearly unmatched. The best reason to join Google is that there's always something to learn, and some of the best mentors in the world to help you learn those things.
You mentioned "internal programs", I just assumed.
By the way, I worked at Google for 5+ years, then quit to do a startup. It's hard, but you're in good company. Word of advice: come up with a clever/funny answer ahead of time, because you're going to get a lot of "Wait, why would you quit Google?!?"
That is tough to hear. I've never met Mike Moritz but he has had a notable impact on my life. I read an interview once where he said that he likes to invest in people under 30. When I heard that in my mid-20s I made it a point to move to silicon valley as soon as I could.
Whenever people would ask why I had this irrational sense of urgency of leaving Chicago (where I was doing my first startup) I would say "Ever heard of this guy Mike Moritz? He said they don't like to invest people over 30 and i'm already 26!" Of course it was me just rationalizing but nonetheless it's the first thing that comes to my mind when I hear anything about Mike. Like Napoleon said, "Everything on earth is soon forgotten, except the opinion we leave imprinted on history"
You can get a lot done (and get wonderful demos) through research vehicles. The issue with sim is similar to the issues with AVs as a whole - getting to 90% is much easier than getting to 99.9999%.
Also, there is this weird thing in AV (and specifically in-house sim) where pointy-haired managers think "if we put 30 SWEs on the problem - we'll make progress!". While raw eng horsepower matters in some software problems, when you are on the line between research and engineering, more SWEs might actually make getting a robust product up harder.
Another thing we've seen is in-house efforts overfit solutions. They don't really have a wide viewpoint (nature of how secretive the space is), and therefore go down paths which are just plain wrong but no one is there to challenge them.