Removing Uber seems to defeat the purpose of this a bit. After all, the goal of venture investing is pretty much "find the Ubers". Most of the "rules" don't actually apply to Uber. For example:
Rule 1 - Uber had no female founders.
2 - Travis and Garrett were "older" founders.
3 - Neither went to a "top school".
4 - Neither worked for one of the name brand companies listed.
5 - Both were repeat founders. If included, FRC's investments in repeat founders would likely perform much better than first-time founders.
8 - Uber is based in the Bay Area. If included, FRC's investments in "Big Tech Hubs" would likely perform much better than outside of tech hubs.
A couple of the other rules might also not apply to Uber (don't have enough data to assess).
On the whole this is a well-intentioned exercise but I wonder if the exclusion of Uber doesn't lead to wrong conclusions.
First Round is a multi-stage firm, but they seem to focus on seed stage where valuations are still in the 7 figures to low 8 figures. At that stage it may be possible to return 3x your fund over 7 years without having super outliers.
The valuation at which you invest dictates the kind of exits you will need in order to satisfy your LPs.
Rule 7 - I could be wrong, but is it possible that Travis is a software engineer? I thought he talked about that in this video: https://www.youtube.com/watch?v=VMvdvP02f-Y starting 3:30. He said "...I started in engineer..."
But isn't this the point of removing outliers, to avoid a single data point overly clouding the significance? To be fair, by similar logic they should arguably remove one or more of the least successful businesses, but all failures are generally 'equally unsuccessful' but no two successes are equivalent.
The whole point of startup investing is to search for outliers. The way returns are distributed in the tech industry, it's not unusual for 1-2 companies to be responsible for 90%+ of a fund's financial returns.
Including Uber would probably have made most of the data meaningless - since their conclusions are valuation-weighted, their data would show that the ideal startup founder is...Garrett Camp. But then, that's how the startup investing business actually works - your data is useless unless you find the one outlier that everyone else missed.
Edit: It occurs to me that this effect could be overcome by taking the log of valuation (or whatever metric is of interest) and then running your statistics over that. That's standard procedure when trying to do statistics over a Zipfian or other power-law distribution; it lets the outliers count, but prevents them from distorting the averages too much.
The mean (or average) is a good choice for data with a normal distribution. However, if your data has extreme scores, such as the difference between an Uber and everyone else, you should look at the median or 90th percentile, because it's much more representative of your sample.
Median and 90th percentile are still pretty meaningless for the question that First Round is asking, notably "If I want to maximize my financial returns, what qualities should I look for in founders?" Miss that one company at the 99th percentile, and your return could be 10x lower.
if you remove a handful of outliers (jesus, buddha, mohammed, joseph smith, maybe two dozen in total) then nobody has ever formed a religion that got any significant number of followers.
In other words, if you remove the outliers you're now looking at something basically meaningless, like evaluating a McDonald's meal by drinking the soft drink only - everyohe else other than the outliers is the soft drink, and the outliers are the main meal.
Articles like this really bother me because its really unclear what data here is significant (if any) since all of the data is just quoted as a % without standard error and the sample sizes are very small:
e.g. on comparing companies that do better. You could have a data set of 150 companies whose exit performance (or current value) looks something like this (numbers in millions):
5000,1000,500,400,200,200,100,50,50,30,30,20,20,20,15,15,15,15,0,0,0,0,0,0,0 (repeated 6x to get 150 data points)
Now compare that against a data set that is those exact numbers divided in half:
2500,500,250,200,100,100,50,25,25,15,15,10,10,10,7.5,7.5,7.5,7.5,0,0,0,0,0,0,0 (repeated 6x to get 150 data points)
That may not sound that bad, but now add a super-unicorn to each one of those data sets, a $20B exit. Now the differences aren't even significant at 80% confidence.
e.g. in Item 7 about technical co-founders: "consumer companies with at least one technical co-founder underperform completely non-technical teams by 31%"
Lets say that First Round has 150 consumer businesses and we're just going to look at a binary outcome of something like "valued over $50m". Now lets say that 100 of these consumer companies have technical co-founders and 50 are completely non-technical. Say 40% of the non-technical teams are "successful" by the $50m metric. That means that 30.5% of the technical teams are successful (if they are doing 31% worse by the numbers in the article since 40%/1.31 = 30.5%). That's not a significant result at 80% confidence (http://www.evanmiller.org/ab-testing/chi-squared.html#!31/10...)
I understand why they published the piece and think it will get a lot of reads, but really wish I could read a version with statistically relevant insights instead.
Given they only have 300 data points and they have tested at least 10 hypothesises it is almost certain that they have been engaged in a massive example of data dredging [1]. The chance that we can draw anything from this study is next to zero.
If this is the sort of data driven work that VCs generate then it might be best that they just tell founders to be data driven and not actually engage in it themselves.
"Data-driven" as in "we looked at some data and made a decision." Whether that was good data or bad data, lots of data or just a couple points, is out of scope. :p
The data they are looking for is can you show more progress so they can come in as late as possible at the lowest valuation. I don’t blame VCs for doing this, but I wish they would be a bit more honest about what they are doing.
The methodology matters a lot here: Were a set of preselected questions answered by the data, or was there exploratory analysis of the data which uncovered these results? If the latter, the effects of data fishing[1] would largely invalidate the conclusions.
Really cool of First Round to share this data. Whenever I see interesting data it makes me ask more questions and these are some of the things I'm wondering about after reading the post.
Female founders outperforming male teams: My hunch would be that the bar for women to get funded (at least historically) has been higher than men so the female led start-ups would be a better calibre of company. Related, since this is based on investment performance, could it be that the female founders received smaller initial investments so performing on par with male teams would make the ROI look better?
Halo effect: This to me would indicate that we shouldn't be encouraging fresh college graduates to work at start-ups and instead get experience at a more mature company. I wonder how much tenure they had at their halo company prior to founding the start-up and how it ties with the average age of founding.
Solo founders perform worse: I wonder what happens if you frame this from the point of view of the founder. If the solo founder had a $100 return and the team had a $260 (160% better) return; assuming equal dilution and equal division between founders, solo founder get's $100, a two founder team get $130 each (30% better), a three founder team gets $85 (15% worse).
Next big thing from anywhere: Also interesting, I'd like to see how this varies by referral source. Do companies referred by other investors perform better than non-investor referrals (or can other investors pick companies better than social connections).
I think we should be encouraging college students to get that experience at mature companies earlier -- ideally as soon as the summer after freshman year. College students can do just fine without summer vacations. That way, by the time they graduate from college (or hit third year and drop out) they'll be poised for success (assuming that a successful startup is your definition of success).
Interesting angle on female founders. My initial thought was "I wonder if they're a much smaller % of the overall pool and so vulnerable to the 'Uber' effect of one or two big exits dragging all the women up" But yours is a great point too. They face more obstacles and so it's a higher calibre out of the gate getting funded.
I think the the things they highlight are all attributes that define hungry/ambitious people, and/or they correlate to things that would hold people accountable and keep them on track.
For example:
Ivy League School and working at a prestigious company? You don't get either of those by being a slacker.
Younger team, woman co-founder and more than one founder? You better believe there is going to be more pressure to prove yourself and not sell early or give up (vs. being a single founder or an older proven founder).
Standing out from the crowd at demo day or getting noticed out of all the noise of social media? That takes some dedication. I guarantee that the people who did get noticed that way didn't just send one email or one tweet. They were hustling their idea hard.
Great read though. I loved the point that startups don't have to come from SF or NYC to be successful!
Let's go with Ivy League gives you connections. My alma mater has pitch events that are only open to alumni and the judges are active investors who are alumni. There is feedback, mentoring, networking, and money flowing in a system that doesn't even pretend to be open to the public.
I think that if someone is a legacy at an Ivy league school (in the sense that you are implying) then that person probably isn't trying to lead the lifestyle that comes along with launching a startup.
Exactly. This looks precise but is not accurate. As some other posts indicate there are many other correlations here. I would love to see the underlying data they use. Good effort but disappointing results.
This seems to be a lot of confirmation bias and use of data that is likely not significant.
For instance:
"The results were stark: Teams with more than one founder outperformed solo founders by a whopping 163% and solo founders' seed valuations were 25% less than teams with more than one founder."
How many of the 300 investments in their portfolio were solo founders? 10?
Solo founders are rare, and it's often harder to raise money as a solo founder. That means less companies have solo founders to begin with.
The 25% less on seed valuation is actually a good news for solo founders. The seed round is just a bet on the team, so if a team of 2, 3 or 4 people are worth "only" 25% more than one guy/girl, then this one guy/girl is doing something very right (as well as optimizing his/her financials, because they have 25% less than people who diluted themselves at least 50% more).
Obviously there are three kinds of selection bias here: companies which raise money; companies which approach First Round; companies selected by First Round. So it's possible these aren't overall trends, but specific to this set.
I think that probably explains the "no tech cofounders do better" bias in Consumer; the bar is probably higher there.
There's also a form of selection bias where many consumer companies without cofounders acquire them before talking to VCs, because they know they're better off with one and it's relatively easy to convince a friend to join you when you can demonstrate a good idea and market demand, which is what the VC also looks for. DropBox, Google, Apple, Instacart all acquired cofounders in-between the prototype being written and them taking venture capital.
> I think that probably explains the "no tech cofounders do better" bias in Consumer; the bar is probably higher there.
For consumer startups often the most important skillsets are social science and product marketing, but that doesn't mean knowing how to code causes you to perform worse; the benefit of being technical is likely exactly the same as with enterprise startups.
Single non-technical, first time founder, 33, no ivy league degree, non-technical, no big brand experience searching for a co-founder.
You should be a former Google or Apple employee, repeat founder, ideally technical, with an ivy league degree,active on twitter with no connections to First Round Capital.Oh, and you should be female, and 16 years old.
We're gonna build the next Unicorn.
----------------------
Seriously,
This is the type of irresponsible analysis that sinks ships. Fun to consume...But causes serious damage because it is complete non-sense, but everyone exposed to it will believe it...even with knowing it isn't accurate...it will still influence people who read it long into the future.
>it will still influence people who read it long into the future
I can’t see anyone with any understanding of statistics being influenced by this “study”, but I do agree there is a real risk that the less skilled VCs might be influenced by it, but I thought you weren’t supposed to accept dumb money anyway :)
I think that's a risk if there were only 3 or 4 big conclusions but with 10 I think nearly everyone gets to stay "Well we hit 4 out of 10 and missed 3 out of 10 with a few in between"
What an awful title to the first insight "women are winning". Its that sort of clickbaity, reductionist and divisive language that's turning gender relations in tech into such a warzone. Where's the data for all-women teams vs. all-men teams? Only then could you draw such a conclusion, if you even wanted to in the first place. Total garbage, and damaging to gender relations to boot. A more constructive (and appropriate, given the data) conclusion would be "we work better as a mixed team".
"We also looked at whether the college a founder attended might impact company performance. Unsurprisingly, teams with at least one founder who went to a “top school” (unscientifically defined in our study as one of the Ivies plus Stanford, MIT and Caltech) tend to perform the best. Looking at our community, 38% of the companies we've invested in had one founder that went to one of those schools. And, generally speaking, those companies performed about 220% better than other teams!"
I'd love to see an inverted analysis of this effect, ie. which schools had the best indication of success. Pre-deciding to look at their definition of "top schools" is probably only seeing part of the picture.
umm.... I think a major oversight here is leaving out IIT founders. The valley startup and venture scene is rife with IITians and there's a pretty tight network effect at play for founders/VC's etc. there.
Could some of these be attributed to selection bias that ends up just confirming the conclusions/lessons learned? Prime examples - the ones re: age, schools, former employers, and repeat founders. Would it be fair to assume that they have a greater bias to fund companies with founders exhibiting these attributes to begin with?
not necessarily, if you're picking better-than-average apples, why would the outcome be worse than for other categories? also consequently, your conclusion would of course be, these apples > other apples. I'm not saying the data is bad or calling them out. I'm just saying some of their lessons learned might just be a consequence of their selection bias.
I think the headline for #1 is a bit misleading. I'd infer that teams with a female founder doing better than ones without one does not correlate to "Female Founders Outperform Their Male Peers." I'd think a better title would be more along the lines of, "Diverse Founding Teams Outperform All-male Ones."
We don't actually know if that's what the data shows. For one, diversity can refer to age, race, national origin, etc. They only refer to sex, all those other forms of diversity could actually be a negative for all we know.
Additionally, it's possible teams with 100% female founders do better than ones with a mix of sexes among the founders. That would mean again that diversity is not good, but being female is.
Obviously though, this is an n of 300. I would guess that includes less than 100 companies with female founders, making it not exactly proven.
Yeah, but what's more important: Accuracy or the amount of attention this listicle gets?
Women > Men = more clicks. All kinds of places that wouldn't give this post the time of day will talk about it. People promoting feminism/diversity/what have you will cite it for years.
This analysis is probably way too sensitive to investment price and all sorts of hidden causal relationships.
The biggest problem is that "performs better" is thrown around a lot but never defined. My hunch is that performance is measured as a return on investment. For instance if First Round invested in a company at a $5M valuation and the company is now worth $100M, that's a 20x return. If they had invested at a $10M valuation, the return would only be 10x. So I suspect in their eyes the "performance" of the first investment is better. Could be wrong, but my point is there's no way to know.
Without knowing that it's hard not to look at their conclusions such as young founders do better and repeat founders cost more with a large helping of grains of salt.
A young founder is less likely to be a repeat founder. Therefore these founders will cost less, per the conclusion they reached. And if performance is measured as ROI then they will "perform" better even if underlying talent or company growth is constant.
So it looks as though there's just a lot of observational data without much insight.
I'm not convinced about the age conclusion: depending on which statistics you focus on, you either conclude that 25 is best or 32 is best:
Founding teams with an average age under 25 (when we invested) perform nearly 30% above average [...] for our top 10 investments the average age was 31.9
If your goal is to be a top 10 investment, 32. If your goal is to get funding, 25. Most founders, I would suspect, especially at First Round's target, are happy to get investment period, even if they aren't in the top 10 (yet).
I think a lot of these figures are correlated. Multi-company founders command higher valuations, who will tend to be older than their first-company peers, so it makes sense the top 10 list is slightly older on average.
My guess was to make a Top 10 investment you had to probably be in business 5 - 10 years thus invest at 25 and by 32 they have a very successful startup/exit
A lot of people have concerns about the methodology here, but I haven't seen this really big one addressed: what does "perform" mean? Are we talking about the performance of the startup, or the performance of First Round's investment? If we're talking about the performance of the investment, then these stats are skewed towards those founders who are willing to accept lousy terms.
The comments here make it clear that I am not alone in my skepticism regarding their conclusions.
But to be fair, at the bottom of the page, they say that they are not trying to claim any statistical significance... just trying to look at their own data to gain some insights. It doesn't sound like they expect any of this to be taken too seriously.
No wonder the results skew towards elite education, they measure performance by market value. Your market value, under a certain threshold in terms of company size, is dominated by your network. Which the top schools are very good at providing. I wonder what difference would it make if First Round analysed revenue and profitability instead of market value.
How many minority non Ivy League startup founders did and have they funded?
The majority of their team consists of white guy ivy leaguers who have a penchant for funding their younger counterparts. No offense to first round as that is how the VC market looks and resembles
Until that goes away the diversity issue still stands.
It's unclear to me what "did better" means here? Is it that they raised more money, perhaps at higher valuation, OR are they looking at exits. I hope it's the latter and not the former since raising money and/or valuation is not success.
Given that none of them were +100000% it's all basically noise, right? If you can successfully identify 100x or 1000x companies some percentage of the time you're a good VC. If you can't you either get lucky or go out of business.
Everything else is secondary to finding the home runs. Even if you multiply all those advantages together you get:
So if you manage to somehow get every single one of those attributes at max in a company you'll get roughly 66x the valuation or performance or whatever versus a company/team with none of them.
Of course if you go for all those things you'll probably only get one deal per year.
They didn't say that these factors were independent and causative. In fact there is likely to be a high correlation between a lot of the factors with high value. What would be more interesting is if they made a model of success with all ten factors and published the coefficients.
I wish I could see these factors ranked against other metrics, eg. # users, revenues, profitability, exit size, % exited. Their key metric is valuation, which makes sense when judging your performance as an investment firm but is relatively useless to a founder. And their top 3 factors (has cofounders; brand name school; brand name employer) are all things that investors value highly, probably more highly than customers. I'd love to see whether the magnitude of the effect of each of these remains true when ranking companies by more founder-focused or customer-focused metrics.
While the data is interesting I don't think it's very useful. For founders, many of these are things you can't change. One of the few actionable points for founders is about having the right kind of cofounders. For VCs this data doesn't tell them what they really want to know. VCs want to invest in outliers. They say right near the beginning of the article that they removed Uber from the data. Actionable data for VCs would have info to help identify potential Ubers or AirBnBs.
Just looking at data and making assumptions is no better than astrology. For a supposedly scientifically literate field, tech makes use of a TON of pseudoscientific hand-waving.
How much of this is self-fulfilling? Ie companies with certain qualities do better because they get funded, and they get funded because VCs think those qualities matter.
I wonder if the outperformance of organic picks (non-referred companies) is a selection effect. That is, VC prefers to invest with a referral, but will go in on a non-referred investment only for the most promising opportunities.
Rule 1 - Uber had no female founders.
2 - Travis and Garrett were "older" founders.
3 - Neither went to a "top school".
4 - Neither worked for one of the name brand companies listed.
5 - Both were repeat founders. If included, FRC's investments in repeat founders would likely perform much better than first-time founders.
8 - Uber is based in the Bay Area. If included, FRC's investments in "Big Tech Hubs" would likely perform much better than outside of tech hubs.
A couple of the other rules might also not apply to Uber (don't have enough data to assess).
On the whole this is a well-intentioned exercise but I wonder if the exclusion of Uber doesn't lead to wrong conclusions.