HN policy for years has been to allow two kinds of job ad: (1) job posts by YC startups, and (2) the Who's Hiring threads. We kill job ads that are neither of these.
(Incidentally, PG designed the /jobs posts to be the one privilege that YC startups get on HN. Other than that, the site policy is to treat them the same way as everybody else.)
It's not uncommon for posts to be in a grey area between job ad and story. A programming test for an interview process doesn't seem all that grey, though, so I've been killing these. Sorry to be a hard-ass about this, but I don't want to be accused of enforcing the policy erratically.
Edit: I'll unkill the post (but otherwise leave it demoted) so the discussion can remain open.
Edit 2: The argument has been put to me that this is newsworthy because A16Z are doing it. Since that's corroborated by external sources (e.g. Twitter), and the post has gotten a lot of upvotes even while it was buried, that's reasonable—so we'll restore it as news rather than as an ad. Besides which, HN is one of the few places where a programming test counts as interesting.
Can an actual list of policies be posted somewhere ? - I don't ever recall there ever being a discussion of such a policy before. Certainly there have been highly voted non-HN job posts in the past (e.g. Khan Academy)
I'm reluctant to mess with the informal way that HN has been run over the years, so there isn't likely to be a complete list of rules. We do intend to augment the HN guidelines, though—just not any time soon. The new team has to get settled in first.
In the meantime, I've been posting feedback comments most places that issues like this come up. You may need to look to the bottom of the threads to see them, because we usually mark them off-topic.
And you can always get answers to questions by emailing hn@ycombinator.com.
The informal way things worked was by the community not upvoting posts or in some cases flagging them; if the new HN position is that moderators will manually intervene to enforce these informal policies then that's the major change. Publishing the policies is minor in comparison.
The informal way things worked before is exactly the same as how they work now. There is no "new HN position". The only difference is that we're trying to be more transparent, and that's not a policy change—it's that we have more time to put in to answering questions, and users made clear it was something they wanted.
I wonder if this could act as a negative filter for good programmers. Who worth their salt is going to invest 3 hours of their time just to submit a resume? Wouldn't you want to talk to the employer first to see if they're a good fit?
In a tight labor market, people willing to do this are more desperate than others. So it's going to filter for desperate developers who passed a 3 hour basic skills test.
Old fashioned prior experience and education is probably still a better first filter. This would probably make more sense specifically for junior hires right out of school or with no experience.
Well, empirically we're getting quite a lot of people attempting it! :) Three hours is pretty lenient (you could blaze through it in less time), though the last question is challenging. We think the questions are kind of fun as well.
Without something like this we'd just have 100s-1000s of unstructured resumes in different formats to rank against each other. This process is more fair and (IMO) allows someone without a college degree, or from outside the US, to compete on a level playing field.
> Well, empirically we're getting quite a lot of people attempting it!
> Without something like this we'd just have 100s-1000s of unstructured resumes in different formats to rank against each other.
So instead of hundreds or thousands of unstructured resumes in different formats, you have hundreds or thousands of completed programming tests. Even assuming that you automatically filter out submissions that fail testing, given that there are a lot of ways to solve a problem, you'd still be stuck evaluating potentially hundreds of correct submissions.
More importantly, a test doesn't seem to be aligned with your top priority:
> ...but the most important requirement is significant independent programming experience as demonstrated by your GitHub account, personal projects, academic publications, or startup success.
Are you assuming that all of the folks most accomplished in the real world are going to complete your programming test successfully?
Since you ultimately seem to be interested in folks who can build dashboards, I would think you'd be far better off asking candidates to build a dashboard prototype using sample data you provide. There are lot of really smart developers out there with advanced degrees who could solve programming quizzes all day long but couldn't design and implement a useful dashboard application.
> You'd still be stuck evaluating potentially hundreds of correct submissions.
I'm guessing the MVP of hackerrank was the web code editor + lint/findbugs/whatever for filtering. A lot of people have spent a lot of time writing open source projects that convert code to metrics, so making a reasonably prioritized list is probably fairly easy.
> There are lot of really smart developers out there with advanced degrees who could solve programming quizzes all day long but couldn't design and implement a useful dashboard application.
Agree!
I think this is a great critique of HackerRank as a business idea -- while they are new, it's just a hurdle that weeds the busier coders out of your hiring pool... if they become established there will be books on cramming for it, and will no longer separate the wheat from the chaff.
To be honest I think it's the other way round - if you can do programming tests all day long you will succeed at most coding jobs. However there are a lot of developers who can't tackle an interview that with a nights sleep and no pressure can easily code a intuitive and effective architecture.
Sorry - programming tests raise far more false negatives than false positives (and this from a person who considers himself a false negative in most tests)
I'm not sure what kind of coding work you do, but I assure you the vast majority of web development work has little resemblance to coding questions. You are almost never designing an algorithm: that was already done for you by whoever designed the library you are using.
Majority of the tasks IME:
- code reading & refactoring
- figuring out why two libraries aren't playing nice
- bug triangulation
- optimization (not the algorithmic type, but rather the fix-the-auto-generated-SQL-query / frontend asset packing kind)
- repetitive/simple/obvious bits of jQuery
Maybe there's some kind of online test that could exercise those skills, but if you ask me the main performance changers for modern web development are, they're: really knowing the stack (problem anticipation, what to use, how to use the various tools to quickly get search keywords), google fu, and most importantly having done it a couple hundred times (muscle memory).
That's terribly negative. If you are doing agency work maybe (same website, same functionality, just different skin) but that vast vast open plains of http-based development are around and going to be around joining parts of the business, public world together in ways they do not know or imagine.
Yes, binary trees are pretty much coded for you, and you would be a fool to roll your own hashing algorithm for production, but thats still not a reason to keep your head down.
I don't understand this response. To be clear, I think web development is fun. I very much enjoy it and don't resonate with "keeping my head down".
I guess to reiterate my point: once you've done web development for a couple years, coding most websites is less about solving new problems and more about applying known solutions.
To reiterate how this point relates to the question of coding interviews: in as much as they don't test for lots of known solutions, and the ability to apply known solutions quickly, they aren't filtering for productive coders.
How does your response relate to the points at hand?
but web development is not about "making new websites"
Think micro-services supplying REST APIs over HTTP - that's going to be same-y after a few years yes, but behind that is a world of business apps, integration challenges, complex algorithms and calculations - sorry I just see "web development" as "anything that has of will have an http access point" which I mentally count as "pretty much everything from here on in"
Programming tests do generally test for ability to apply known solutions quickly - it's about all they do do. You just have to know the known solutions (oh look calculate primes).
As for finding productive coders - well the amount of false negatives is large, and the false positive rate is probably higher (plenty of people who can waltz through these tests i find fairly poisonous to morale) so I am not a big fan - but if you need to make a first cut this is a good start
> Well, empirically we're getting quite a lot of people attempting it!
I think you missed my point. It's that the people submitting to a multi-hour exam just to submit a resume are more desperate / value their time less than those that can stand on their experience.
The fact that there a "quite a lot" of submissions only worsens the probability that the multi-hour effort will be wasted.
The questions were pretty good, because they were close to the real world and you couldn't google them so easily.
The programming tasks were rather unbalanced. The first two were super easy, while you can easily get stuck on the last one, even as a good programmer. I knew that I wouldn't be able to answer it today, so I skipped it after a short while.
If I'd plan such a test, I'd try to get an example of the structure and comprehensibility of my future employees work.
>> allows someone ... from outside the US, to compete on a level playing field.
Just curious, why do you say that the normal process doesn't allow people outside the US to compete on a level playing field? Higher barrier to on-site interviews?
one of the best programmers i know refuses to continue with a job application if there is a programming test... its a bit eccentric but valid given that his experience and past achievements say more than a programming test ever could.
I have come across this opinion as well from some excellent developers, but I can't say that I agree with it. All in all, a well-executed* technical test can be an excellent filter and basis for further interviews.
* By well-executed, I mean - relatively language agnostic, lacking in trick questions and meaningless trivia, and being open to multiple good answers. The intention is to not only check basic ability to solve simple problems, but the general approach and programming style of the individual interviewed.
Past experience is often vaguely stated in CVs and looks great on paper, but can be disingenous. It takes a thorough two-way discussion which is customised to each individual to reveal this, however. Some people have excellent and highly visible past achievements, but not everyone does. I wouldn't throw out a CV simply because the individual doesn't have any open source code to show me, nor do they have any awards from companies they have worked for (which is often more about self-marketing than actual ability).
I guess your mileage may vary based on the position that's in question and the companies involved, but I've seen some shocking discrepancies between CVs and actual programming ability. Even worse than the usual "fizz-buzz" statistics that get thrown around.
this is something i wonder. are tests to see the approach to problem solving? of the few tests i've had one was simply testing my knowledge of the C++ and STL specifications - there was no programming skill involved at all - I could have cheated if I had a physical copy of the spec handy even.
this kind of test is not valuable. the other tests i've done though - even though incredibly easy i think would be more useful for seeing how i approach problem solving in general...
My own surprise was mostly to do with highly paid individuals with a great track record on paper, but who seemed to be barely able to solve basic problems (to the level of making me wonder whether they had actually written a single line of code in the past few years).
Regarding CS majors, there's always the possibility of an individual being great at the theory part, but not having much programming experience. This should be rare though; during my own CS degree, we coded so much that I was frequently dreaming about code. If it's provided in the CV, I like to scan through the subjects list of the candidate - the ones who don't actually like programming seem to have an aversion to taking CS courses for elective subjects. As always though, YMMV, and it's a good idea to keep an open mind.
why does it amaze you? my experience of 'self taught' individuals in the general case is that they have a rich understanding of the subject by necessity.
conversely my experience of education is that it trains you to solve education style problems - which are essentially absent from the real world - where being able to invent and implement a slightly broken solution in an hour with 30 seconds notice is much more valuable than being able to solve a problem you know about far in advance and are given loads of information and guidance to help in preparation for it.
on the other hand if i wanted someone to do research then a CS major might be a better choice. :)
Why? I've had that in interviews before when asking people to code. I find it strange (not just because we pre-warn about the agenda) that someone applying to program won't...program? I reasoning I've had has been mainly : "it's too trivial" "I've written gems" "I'm famous"
I am 26, been out of school for 5 years. I've been programming since 14 years old, and currently know 5 different stacks by heart. I would never do a programming test, because I know I would feel pressured to do well (due to my experience), but also I would fail horribly at it. It sucks. :(
3 hours is upper bound and extremely friendly one to allow for all but hopelessly incompetent candidates. I did the test in less than 90 min. First 27 questions should take a competent hacker less than 20 min; rest of the time is for last 3 coding assignments.
I'm saving it for the weekend. Will read about all the virtualization techniques for those spoj like sites and then have a go in 3hours trying to break that one :-)
This doesn't quite make sense to me. Or, it makes sense, but not for the reasons presented on the surface.
If you're Andreesen Horowitz, you have access to a nearly-unlimited supply of capital. You also have incredible access to information and dealflow throughout the industry. In situations in which you are not capital constrained, the most efficient way to establish a workforce is to acquire it from somewhere else; that's why acqui-hires make so much sense, because you've basically had someone else do the hard (and often completely unpredictable) work of figuring out if people can work together in a successful manner (which, I should note, is different from determination of whether the thing they're working on is successful). If I were A16Z, I would arbitrage my information advantage to secure a high quality engineering team by acquiring a company that had smart people but a lame product; going through a traditional hiring process would be a waste of my time (and since everyone is throwing money at my firm, it would also be a waste of time that could be better spent on figuring out where to deploy my capital).
A16Z is rich enough to buy people, if it really wants people, for internal use (whether it is for their own company needs, or for some idea that they want to incubate inside of the firm). And even if they don't want to buy a whole set of people, they have enough wisdom and experience to know that you want to hire within a cohort of successful people rather than randoms (to see this in practice, take a look at Don Melton's post about Eazel alumni ending up at Apple on the front page right now). This tells me that their programming test has more to do with some sort of evaluation of this Hacker Rank service ("tell you what, we will run our own test on the service to see what happens!") or some other sort of analytical adventure.
Or, you know, maybe it is really a programming test after all. Maybe they are hoping to find some sort of diamond-in-the-rough engineers who are socially and career-wise disconnected from an existing network. Some sort of "black swan programmer" or something. But I highly doubt it.
So basically, if you go through this test, you're helping A16Z evaluate something; that something, however, probably isn't you.
Even Andreessen is unlikely to be structured such that spending millions of dollars at a time on things that don't make returns is a sensible thing to do.
You have to understand the way capital inside venture funds is usually allocated - it's not just one big checking account. A common way to describe the economics of a venture fund is "2 and 20" which means that the fund takes 2% management fees per year (across a 10 year fund lifetime) and takes 20% of the returns or upside.
So, in a hypothetical $300m fund with this arrangement, 80% of the money is reserved for making investments that provide returns. Every time an investment is made, a capital call takes place where the LPs are asked for the money.
The other 20% of the fund goes to management fees - 2% per year although it's not uncommon that these fees be non-linearly distributed, IE 3% in the early years and 1% in the out years when (presumably) new funds are also throwing off management fees.
So your $300m fund really generates $500k / month in management fees. (Probably paid annually or quarterly.) Enough for some very comfortable salaries, ample support staff, great office space and a travel budget but not enough to really start throwing dollars around millions at a time.
Now of course Andreessen has a reported ~$4B under management but I'd be surprised if they're really generating north of $6M in management fees per month. As fund size grows, management fees may come down (since it doesn't take 10x the effort to write checks 10x as large) or a fund as confident as Andreessen (and in as strong a position) might make a "1.5 and 30" deal where they take fewer dollars as management fees up front but get a larger percentage of the upside.
Note, I know absolutely nothing about what's really happening inside Andreessen, this is a purely hypothetical story against what is considered "typical" in the industry. (In the US.)
I think you're reading too much into this. Sometimes a programming test is just a programming test.
If I were to guess, they're building an internal dashboard and DWH to crunch numbers on their investments (or perhaps potential investments). This test appears to focus on that skill set.
Yeah, they could probably buy a company, but that would be overkill. They're VCs, they're supposed to be selling companies at a premium, not paying premiums. ;)
Why acquihire someone who knows how much they can ask if you can pay close to nothing to that test website (it's a genetic service) and get a clueless buy competent nerd that will be glad to work alone on a project for peanuts?
That's why we're poor and they have "unlimited capital"
There are a lot of stupid rich people in this world, it's true. It's also somewhat of a dirty little secret that some of Silicon Valley's most successful VCs are stupid rich people.
The team at Andreesen Horowitz, however, are not among these stupid rich people. If anything, they have disrupted the standard operating procedure among their peers by outsmarting them, and the nature of how they have outsmarted their peers is strongly connected to an appetite for attaching high valuations to small groups of really smart people. Their organizational M.O. has been the exact opposite of bargain-bin talent hunting, which is why this whole Hacker Rank thing comes off so weird to me.
I want to be clear that I don't think A16Z is up to anything evil. Just something sneaky (which is fun!). These are smart people, and for them to publicly engage with the hacker community in this manner is indicative of some sort of really smart hypothesis being tested.
Smart rich people are the opposite of cheap. They have more money than time, know it, and are perfectly happy to trade one for the other.
Building organizations is expensive, unpredictable and takes a long time. Buying organizations is more expensive in the short term, but also far more predictable and much faster. So if you have more money than time and need an organization to get something done, buying one is far more rational than "paying a clueless but competent nerd peanuts".
I'm not saying a16z is doing that here, btw. I'm just saying that you're wrong.
Not to sound offensive , but you're not A16z.
1.<< A16Z is rich enough to buy people>> No one is rich enough to buy people , unless well if you're talking about criminals.
2.Most good engineers will tell you , that money is NOT the primary motivator for them to build great products.( So aqui-hiring won't work)
3. As for Hackerrank--> that's what they're supposed to do - find people with good technical skills.
Yes every allegiance has a price , however What you're suggesting from what i understood is that A16z would pay 50 million dollars to a new hire. With them being in the VC business , i think they know that , such amount of money can possibly invested in growing their portfolios.
I don't have any evidence of any engineer being offered such exorbitant salaries , however this gets close.
Ok, it's somewhat of a hyperbole. 50M for a single hire is extremely exorbitant (after all, we're not european soccer players), but consider being offered a salary 5 times your current pay.
I'm loyal to my employer as hell, even though he only pays me around 45k (euros, yes, i should get more ;) and I'd like to think I'm 'morally gifted', so I wouldn't consider quitting my current job when someone offers me double... 3 times plus, is a different story, I'm scratch my head at least twice.
Remember that VCs have money to spend on acquisitions for their investors, not for themselves. It's very rare to see VCs acquire talent for themselves. It's usually personal network based hiring. What makes this unique is that it's a talent test first - groundbreaking in an industry that normally hires based on personal connections.
The real question is "What problems will these folks be solving?"
An entire page detailing what you want from me, coupled with a 180 minute test. Yet in all of this there are only 4 bullet point sentences describing what you're offering me. That deal is not for me.
So, someone taking Statistics 101 is considered to know machine learning in the startup circus? (Not that this would really surprise me, given the tendency to appropriate terms that had a certain meaning for purposes that have only a passing similarity to the original meaning.)
As an employer, I would seriously hope this does become the norm.
If some employers want to implement recruiting processes that require developers to complete tasks that don't reflect real-world development, it makes it that much easier for other employers to differentiate themselves and attract competent developers who aren't interested or desperate enough to play these games.
Frankly, in today's market I feel sorry for any experienced developer who is going to complete a "programming test" in which referencing a website or book other than language documentation is forbidden. I reference websites and books all the time, and I don't know a single developer who doesn't.
This was too awesome a comment to pass up, so I ended up browsing your thread history / googling you. I found a lot of great posts as well as the hilarious https://news.ycombinator.com/item?id=5967671... but sadly no real name / business name.
What company do you run? Are you in the Bay Area? Can I buy you coffee? I have a feeling you could teach me things that are both true and that hackernews would downvote... And that there's a reasonable chance I could return the favor.
Background so you can filter me more easily: I'm a very good programmer (7 years experience coding on small teams made of up jQuery core, rails core, and MIT AI Lab alums. Several offers from YC companies. 4 papers & patents. Currently doing machine learning for the Scripps Institute of Oceanography). I'm also an avid reader of psychology (cognitive biases mostly), marketing (Joe Sugarman is amazing), and business books (I'm a raging Munger and Deming fan).
So I'm guessing if we did meet for coffee you at least wouldn't find it boring.
Seems like the obvious (if dishonest) thing would be to use a throwaway phone # to get the test, then spend a day or so solving it, then type it in (with plausible delays) using your real number a day later. Assuming the test is roughly the same each time (even if details are different, it's probably of the same general class).
In grading, the results of 6-8h of work entered in 2h would look more impressive than 3h of work in 3h, I suspect.
(Sometimes I hate that whenever someone gives me a situation, my first thought is "how can this be cheated or gamed", followed by "what controls can I put in place to prevent that, and and what costs"; it's basically automatic.)
Credentials through phone # is more of a deterrent. We're also going to have an in-person coding exercise as a next step, which would be much harder to game :)
Why would you want to appear better then you are at the interview? You'll get found out eventually later, and this will end up just spending your time.
There are plenty of developers/programmers who do not ace tests with rigid constraints, but are quite competent in a more relaxed setting.
That is they are not the 1% that the all hirers, but they are maybe top 20% and that is actually what most jobs require no matter the delusions of hiring managers.
Granted this particular job might be an exception but I doubt it.
Probably to build internal tools. A lot of VCs firms are trying to apply more data to their investment decisions and that requires internal tools to handle it all. Also makes sense given the qualifications they are asking for.
Are you suggesting that a16z doesn't do this? Or that their firm is not doing well? It would seem to me that their limited track record is ... more than adequate.
Similarly, I would infer that your suggestion is ignorant that they have interest in the companies post-funding (as cnaut suggests "A lot of VCs firms are trying to apply more data to their investment decisions and that requires internal tools to handle it all"... which I think would suggest that they would want to examine the results of said decisions as well (which goes beyond simply the submission process)).
(1)
It appears, really is quite clear, that
they have high contempt for initial contacts from
entrepreneurs via e-mail. Instead they want to believe
that the 'good' entrepreneurs will have enough 'exposure',
enough in 'networking', and enough in 'hustle', e.g.,
on LinkedIn, GetHub, etc., to come to the attention of
A16Z by means other than an e-mail message "over the
transom". E.g., they want an entrepreneur to get
introductions from bankers, lawyers, accountants,
or CEO's of companies in the A16Z portfolio. So,
A16Z (A) wants others to 'filter' their
entrepreneurs for them and/or (B) believes that
lawyers, etc. are good sources of good recommendations.
Nonsense.
(2) A16Z will just refuse to consider any
very technical support for a project.
E.g., if a project has some powerful, valuable
'secret sauce', intellectual property,
and high technological barrier to entry,
A16Z can easily just dismiss the technical
issues, the project, and the entrepreneur.
A16Z wants to believe that they have
'deep domain knowledge' where they invest
and, thus, wants to believe that anything
new that A16Z has not heard of must be
nonsense. If some supporting technical
material is not easy reading for them,
then they believe that they are justified
in just regarding it as nonsense.
But, looking at the partners of A16Z, it's
tough to find one with enough technical
qualifications and/or accomplishments to
(A) get a tenure track position in
the engineering school of a
leading research university,
(B) be an editor of a leading
technical journal of original research,
(C) publish a peer-reviewed paper
of original research in a technical field
in a good or better journal of original
research,
or (D) get an NSF grant for original research.
Net, A16Z apparently is not competent to review
or even to direct a competent review of
technical material in information
technology.
What the A16Z financial performance really
is is likely not easy to discover.
You seem to suggest that the project for
which the OP is some recruiting is for
an internal information system to
do causal modeling and
make valuable predictions
for venture investing. It would
appear that they want to do
cross tabulation and curve fitting;
sorry, there is nearly no chance
those techniques will work. Why?
In simple terms, they are looking
for new, exceptional cases, and
looking at 'averages' from the past,
even of successes in the past, is
just intellectual self-abuse.
If they are making money, then, fine.
But from all I can see, in information
technology their competence is way below
a level that would be useful for
actual technical work or doing
a technical review of such work.
I've been around technical work in
US national security, US business
and start ups, including one of the
most successful venture funded companies
ever, and high end academic research,
hold a Ph.D. in engineering from one of
the world's best research universities,
have published peer-reviewed original
research in computer science, applied
mathematics, and artificial intelligence,
and in comparison the technical qualifications
of
A16Z look like a bad joke.
Yes, most good quantitative analysis
needs some computing and programming.
So, in their recruiting A16Z seems to
believe that the computing, which is
necessary, is also sufficient. It's not.
By an analogy, A16Z is looking for
nurse practitioners instead of
MD physicians. Both are necessary,
but the nurse practitioners are not
sufficient. There is a difference!
"(1) It appears, really is quite clear, that they have high contempt for initial contacts from entrepreneurs via e-mail."
- First, what is your sample size? Do you have any idea how many meetings they give come through email? Just because they may have rejected you (no idea), doesn't mean they reject everyone. Second, I would suggest that a cold email is probably the worst way to approach ANY VC.
"A16Z wants to believe that they have 'deep domain knowledge' where they invest and, thus, wants to believe that anything new that A16Z has not heard of must be nonsense. If some supporting technical material is not easy reading for them, then they believe that they are justified in just regarding it as nonsense."
- I would disagree. That being said I think you're misunderstanding the point of the VC business. It is NOT to create / promote new technologies only. It is to promote businesses and segments that they believe will be profitable (in general).
"But, looking at the partners of A16Z, it's tough to find one with enough technical qualifications and/or accomplishments to (A) get a tenure track position in the engineering school of a leading research university, (B) be an editor of a leading technical journal of original research, (C) publish a peer-reviewed paper of original research in a technical field in a good or better journal of original research, or (D) get an NSF grant for original research."
- Ok. But again you're measuring the success of a VC on technical acumen. What are technologies generally used for? To solve problems. In business domains. You want to go through that partner list again and tell me they don't have experience in various business domains?
"Net, A16Z apparently is not competent to review or even to direct a competent review of technical material in information technology."
- They don't make a claim to. Next?
"What the A16Z financial performance really is is likely not easy to discover."
- Let's see. Google "a16z performance returns". First article: http://finance.fortune.cnn.com/2012/07/23/nice-ira-andreesse... . I must be a genius. Yes, this is but the first data point; however, it's a start.....and based on other more recent data points (oculus, etc.), it may be suggested they're doing well. It's nice that you want to brush how most VCs are judged as successes or not (by the performance of their portfolio) as secondary in comparison to what you want to judge them by.
"You seem to suggest that the project for which the OP is some recruiting is for an internal information system to do causal modeling and make valuable predictions for venture investing."
- I'm sorry you inferred that. Allow me to be perfectly clear. I have no idea what they are recruiting for. I could imagine a lot of projects....the above is something that is certainly not out of bounds
"It would appear that they want to do cross tabulation and curve fitting; sorry, there is nearly no chance those techniques will work. Why? In simple terms, they are looking for new, exceptional cases, and looking at 'averages' from the past, even of successes in the past, is just intellectual self-abuse."
- It may appear like that to you; however, I would suggest it's probably much more complex. They could be performing (for example) some clustering techniques (k-means, k-medians, what not) across the investments. They could be performing a variety of analytics on the talent development portion to gain a better understanding of expectations of the companies and candidates. They could be doing a lot of things...
"If they are making money, then, fine."
- I'm pretty sure that's not the case with you. I'm pretty sure that even if it was shown they're beating even 80% of other VCs you would still be hyper critical of their technical evaluation skills.
"I've been around technical work in US national security, US business and start ups, including one of the most successful venture funded companies ever, and high end academic research, hold a Ph.D. in engineering from one of the world's best research universities, have published peer-reviewed original research in computer science, applied mathematics, and artificial intelligence"
- Do you want a cookie?
"and in comparison the technical qualifications of A16Z look like a bad joke."
- Yet their business and venture capital qualifications appear to be quite good.
"By an analogy, A16Z is looking for nurse practitioners instead of MD physicians. Both are necessary, but the nurse practitioners are not sufficient. There is a difference!"
- They're not looking for MD physicians as the portfolio companies have the technical expertise.
> Second, I would suggest that a cold email is probably the worst way to approach ANY VC.
Sorry they are having trouble handling e-mail. It's an
old technology and relatively easy to work with. E-mail
is just simple text, much like HN posts. Such simple
text works great at HN. Too bad the 'high tech' A16Z
can't work with simple text.
With all their efforts to find good investment opportunities,
they might just look at their incoming e-mail and then
read it. But your statement seems to indicate that they
want to ignore e-mail, and that does appear to be the
case. Ignoring e-mail is bizarre, but they are free to do
so.
For the rest of your post, you miss the big, huge point:
As is commonly asserted, one of the better parts of
a new project is new technology that is powerful and
valuable, intellectual property, 'secret sauce', with a high technological barrier to entry, for
solving a big, important problem. Such technology,
at least in my interest, is
definitely just in the service of making money, the green
kind.
If a project claims to be able to solve a huge problem
so far unsolved, that many others have tried unsuccessfully
to solve, then a crucial, early question for interest
in the project is what the heck is the crucial 'secret sauce' that enables the solution, that lets this
project do what prior projects could not?
Again, of course, the purpose of and interest in the
secret sauce is making money.
I can 100% absolutely, positively guarantee you that
one can write a very carefully written e-mail
with a project to solve what is clearly a huge
problem with overwhelming and obvious evidence
that a company solving this problem will be
worth $x billion for x high enough to be historic,
and outline some of the crucial secret sauce
and have A16Z totally ignore the contact. Call
them on the phone and tell them about the
e-mail, and they will refuse to dig it out of their
inbox and look at it. They will say, "send it
again", and someone might say, "I don't think
that we can do business along these lines.".
"Send it again" is just a manipulation to
insult entrepreneurs by a firm that
does not value contacts from entrepreneurs.
If I get rich, A16Z won't participate.
The importance of their abilities to
evaluate new, powerful, valuable
technology is just that such evaluation is just
crucial for any project where one of the main
features is such technology. As I made clear,
thankfully for US national security, the US DoD
has long been excellent at evaluating new technology.
If the DoD sees a significant problem and
someone comes up with some new, powerful technology
for the first or a much better solution,
then the DoD will listen and, really, in my experience,
for a good case, proceed. My experience is that
apparently mostly A16Z won't; they
won't even look at their e-mail or
listen on the phone. Heck, just finding their
phone number takes some effort.
Biomedical VCs will evaluate such technology.
Information technology (IT) VCs won't. Why?
The IT VCs just don't want to care about
such technology. Instead they want to make
money other ways. As posted by Fred Wilson
at his AVC.com some months ago, on average the
IT VCs are doing poorly. Still they insist
on not even evaluating new technology.
Their feet are locked solidly in concrete
with their eyes and ears closed, refusing
to pay attention to e-mail and phone calls.
Really the IT VCs believe that all there is
to the 'technology' in IT is little more than
routine software development. On average, this
attitude fits with the past. But so far, e.g.,
from A16Z there are only about 15 projects a
year that deserve a Series A. And it's easy to
see that there are only a few Googles or Facebooks
each decade. So, looking at the average projects
in the past is a hopelessly poor way to find the
15 Series A projects for this year or the next
Google this decade. Still, A16Z won't read their
e-mail to find such projects.
The situation is fairly clear: The IT VCs
want to see traction, up and to the right.
Then they want to invest for the big build out
and the rapid growth on the way to going public.
Sometimes they make money doing this;
on average essentially they don't.
For your
> - They're not looking for MD physicians as the portfolio companies have the technical expertise.
No. My remark was in the context of the project of their
OP and recruiting. For the goals of that project
mentioned in this thread, my analogy with an MD is
correct -- they will need one. But they are not
recruiting for one. Instead they are recruiting for
nurse practitioners. E.g., you mentioned cluster
analysis. So, as I explained in terms of the SR-71,
could not evaluate that project by cluster analysis
of the then history of military aviation and, instead,
need to look at the engineering details Kelly Johnson
was proposing. Similarly, cluster analysis stands to
be nearly hopeless for predicting the future of
IT projects and, instead, must look at the details of the
projects. It does appear that the project of the OP
missed this point.
> - Do you want a cookie?
It's really tough to communicate with someone who
works not to understand. Then even something
simple won't come across. The point was,
in really simple, baby talk terms, (1) the
goal if IT projects is to make money.
(2) One of the best possible parts of a new IT
project is new, powerful, valuable, technology
to provide the first good or a much better
solution with a high technological barrier to entry to a big problem where such a solution
will be very valuable. (3) To evaluate such
a project early on, it is just crucial to
evaluate the technology. (4) My background
shows that I am competent both to create
such technology and to evaluate it, and,
more generally understand the potential of
such technology in projects in business.
From that background, no, I don't want a cookie.
Instead it would be good if A16Z would
read their e-mail and evaluate projects
including any new, powerful, valuable technology.
It's not about cookies. It's about
evaluating projects.
I wrote
"If they are making money, then, fine."
and you responded
> - I'm pretty sure that's not the case with you. I'm pretty sure that even if it was shown they're beating even 80% of other VCs you would still be hyper critical of their technical evaluation skills.
Sure. However much money they are making, they
shouldn't pass up making much more.
The US DoD doesn't do this. Neither does
the NIH or the US pharmaceutical industry.
People in US national security,
biotechnology, and research in mathematics,
statistics, and most areas of engineering
would say that refusing to evaluate
technology must be playing with mice
and ignoring elephants.
Besides, US VC ROI is so bad that being
better than 80% of the US VC firms
might well still mean just losing money
for the LPs.
> It's nice that you want to brush how most VCs are judged as successes or not (by the performance of their portfolio) as secondary in comparison to what you want to judge them by.
No, I judge them by making money. Their business
they have chosen is IT VC. Then if they refuse to
evaluate technology, then they are at high risk of
missing out on the next big win of the next
decade. Why is this clear? Not really because
'secret sauce' played such a big role in most or
even any of the big wins in IT in the past 20 years
but because of the overwhelming power of
'secret sauce' demonstrated in US national
security and the biomedical industry.
The NSF funds research in the mathematical
sciences and more mathematical ares of
engineering for a very good reason;
but Silicon Valley (SV) believes that no business
value can come from that work. SV is wrong.
They may be very badly wrong, e.g., miss
a few new Googles.
I wrote
"Net, A16Z apparently is not competent to review or even to direct a competent review of technical material in information technology."
and you responded
> - They don't make a claim to. Next?
Let's see: At their Web site can see
> At a16z, we bet on entrepreneurs who use software to go after the big problems.
Hmm. Sorry, guys, considering just routine software
is a huge handicap in going "after the big problems".
> We believe this is an incredibly exciting time to be a technology investor.
Okay, here they say "technology" and not just routine
software. But do they want to evaluate technology?
Apparently not. Heck, they won't even read their
e-mail.
> smartphone users are expected to grow from 1.5 billion today to five billion in the coming years.
There they go thinking of sectors again. Maybe with all
those new users there will be new problems to solve
and valuable companies to solve them. Otherwise they
are talking a "rising tide raises all the boats"
growth of a factor of only a little over 3 in 5 years,
and that's not very impressive. Broadly, super tough
to get venture returns from growth of a sector and, instead,
need to get such returns from growth of individual projects,
maybe in a rapidly growing sector, maybe not. For such
projects, 'secret sauce' technology can be one of
the best advantages.
> These entrepreneurs care enough about all aspects of their product/service that they want (or need) to innovate in all the areas that touch it.
So, now they also want to "innovate". That's what I'm
talking about where you thought it was just a cookie. But
such innovation needs evaluation. Of course, if A16Z just
looks at the traction, they they can talk about software,
technology, and innovation, refuse to evaluate anything
technical, and just count monthly uniques, page views,
ComScore data, etc. But A16Z says that they are
in "multi-stage" investing, and waiting for traction
now risks being quite late, too late.
Too late? Just do a little of the arithmetic
all of use here at HN can easily do: Pick an
Internet connection with, say, 25 Mbps upload
bandwidth. Pick a server with, say, an 8 core
processor at 4.0 GHz and 64 bit addressing
with 32 GB of ECC main memory and an armload
of 4 GB disk drives. We're talking $1500 in
parts. Send Web pages for 400,000 bits per page.
Send 5 simple ads per page. Get enough
users to half fill that upload bandwidth 24 x 7.
Have the software fast enough and scalable
enough that just one or a few such servers
can handle the load. Assume $2 per 1000 ads
displayed. Okay, let's multiply it out:
So send
25 * 106 / ( 2 * 400,000 ) = 31.250
pages a second.
Then revenue of
31.250 * 2 * 5 * 3600 * 24 * 30 / 1000 = 810,000
dollars a month.
Now, just why does such
a project want to accept seed
or Series A funding?
Has any project successfully
thought this way?
Sure: Early on Plenty of Fish
was just one guy, two old Dell servers,
ads just from Google,
and $10 million a year in revenue.
> If we are right that software is in fact eating the world,
There is considerable question if just routine software
will yield venture returns. One reason is the low
barrier to entry. E.g., China had no trouble
doing something like Google. Neither did Microsoft.
A16Z seems to accept low technological barriers to entry;
that has to be a big mistake, even if they
are making money. Heck, my local pizza guy is
making money.
I'm failing to see why an entrepreneur
with a good project should waste time
with A16Z who hide their phone number,
refuse to read their e-mail,
and won't and even can't evaluate
crucial secret sauce technology.
> Second, I would suggest that a cold email is probably the worst way to approach ANY VC.
"Sorry they are having trouble handling e-mail."
- I would suggest they're not having trouble handling email...their filtering is working just fine. You would appear to have trouble handling rejection (at least I would infer that)
"But your statement seems to indicate that they want to ignore e-mail, and that does appear to be the case. Ignoring e-mail is bizarre, but they are free to do so."
- Allow me to be clear. They are not ignoring e-mail. They are selectively choosing what they will spend their time on. They have (apparently) chosen not to spend time on your idea. Perhaps in the future you should make your emails more clear, you should be more persuasive, and your idea more compelling.
"For the rest of your post, you miss the big, huge point"
- Oh I get it. I just have quite a bit of contradictory evidence in how they work. Let me paint a picture for you. Perhaps you've heard of Prof. Cheriton of Stanford. He's written a publication or two and invested in a successful startup or two. He invested in a startup called AsterData. So did A16Z. Do you see how that works? They don't necessarily need the deep technical expertise within their team. They can rely on others to do that. Similarly, you might recall me mentioning that cold email is not the best course. If the technical idea you're suggesting is so revolutionary, you should have little to no problem having a well-respected professor get you an introduction.
"I can 100% absolutely, positively guarantee you that one can write a very carefully written e-mail with a project to solve what is clearly a huge problem with overwhelming and obvious evidence that a company solving this problem will be worth $x billion for x high enough to be historic, and outline some of the crucial secret sauce and have A16Z totally ignore the contact."
- Actually you can't. That's hyperbole again. Showing a problem and presenting a solution w/o evidence that you'll be able to execute it in a compelling manner would appear to be more of the problem for you.
"If I get rich, A16Z won't participate."
- I sincerely hope that works out for you.
"If the DoD sees a significant problem and someone comes up with some new, powerful technology for the first or a much better solution, then the DoD will listen and, really, in my experience, for a good case, proceed."
- So if some random PhD just emailed Lemnios, he would pick up the phone and call them? Doubtful..sorry. (and yes...i did my time with DISA as well)
"My experience is that apparently mostly A16Z won't"
- And mine contradicts yours.
"on average the IT VCs are doing poorly"
- Fantastic, we're not discussing "on average"...we're discussing A16Z.
"Still they insist on not even evaluating new technology."
- Incorrect. What about Oculus for example? Or do I need to provide more examples?
"Their feet are locked solidly in concrete with their eyes and ears closed"
- You might be looking in the mirror.
> - Do you want a cookie?
"It's really tough to communicate with someone who works not to understand."
- Boy, do I ever agree.
"It's not about cookies. It's about evaluating projects."
- Perhaps they evaluated and weren't impressed. Or is that impossible?
"US VC ROI is so bad that being better than 80% of the US VC firms might well still mean just losing money for the LPs."
- Fantastic. Again we're not talking about the broader VC industry.
"No, I judge them by making money. ... Then if they refuse to evaluate technology ... ...But do they want to evaluate technology? Apparently not. Heck, they won't even read their e-mail."
- You seem to keep missing the obvious point of perhaps they evaluated the technology you seem to be so bunched up about...and didn't care for it.
> - I would suggest they're not having trouble handling email...their filtering is working just fine. You would appear to have trouble handling rejection (at least I would infer that)
No, I'm just reporting some strong evidence that they
just don't read their e-mail. Or, to be more clear,
apparently essentially all information technology (IT) VCs will just ignore an e-mail
message saying (generic version; real version
with specifics):
(1) Here is the problem we are attacking.
So far it is easy to see that there is no
good solution. Fairly clearly a good solution
will be of high interest to over 80% of all the
Internet users, work station down to smart phone, in the world.
(2) The main reason the problem has remained
unsolved is that knowing how to solve the problem
is a challenge. The team has a new, unique,
powerful, valuable, rock solid solution that
has a high technological barrier to entry.
(3) Monetization? With a few billion unique
users a month, for a first cut, just run ads.
For more, there is an aspect of the project
that permits some especially effective ad
targeting with especially good protection of
user privacy.
(4) The crucial core software implementing the
technology is production quality and scalable.
The rest is just an especially simple
front-end Web site, nearly done.
(5) Team qualifications in business,
research, and software are high.
Then with 1-5, they will just ignore the contact.
No response. No questions. Nothing. They just
ignore it. From the claims there, we're talking
$100+ billion exit value; they just ignore it.
No response; just toss it into the bit bucket.
Look, we're talking a billion or more unique
visitors a month and an exit value north
of $100 billion, and they have no response at all.
Strong evidence that they just ignore their
e-mail. Leave a $20 on a sidewalk; a
guy walks by; the $20 is still there;
conclusion, he didn't see the $20. Or,
the VC firm just didn't see the e-mail.
One reason is, US IT VCs just have never seen any such
thing, that is, an IT start up where what is crucial
is the 'secret sauce' technology to solve the
big problem. Instead, they pursue only other
ways of making money and where the technology
is essentially just routine software. US
IT VCs and A16Z just will not give any value to
(that is, won't spend time to investigate)
results of research in IT -- the DoD is
awash in valuable results of such research;
biotech VC will take research seriously; but
US IT VCs haven't seen
any big business success from such research, don't
really understand research or how to evaluate it,
want to pursue other ways of making money,
and want just to ignore anything based on research.
They just want to f'get about research.
That is the obvious explanation.
Since the US IT VCs are just determined to ignore
research, the flip side of that is an advantage
for any project with some good research.
Net, they don't read their e-mail and won't
evaluate technology.
> - I would suggest they're not having trouble handling email...their filtering is working just fine. You would appear to have trouble handling rejection (at least I would infer that)
No, the situation is just as I said: The evidence is
overwhelmingly strong that they just refuse to
read e-mail from someone they don't know. That's their
choice.
> They don't necessarily need the deep technical expertise within their team.
Yes they do or they are at risk of missing out
on possibly some of the best projects. As I made
clear, for the past 10-20 years, information
technology (IT) VCs have not seen big successes based
on anything at the level of publishable technical
research. So, they just f'get about thinking about
or evaluating such work. Again, the track record
and promise of such research is from DoD, NSF, NIH,
etc., not IT entrepreneurs over the past 10-20 years.
Research based projects will likely remain
only a tiny fraction of what US IT VCs get in their
inbox, but such projects are a good bet for a few
$100+ exits over the next decade. Considering
how few such successes there are,
ignoring
such projects, especially with the example of
DoD, is foolish.
For Cheriton,
they were not reading just some e-mail or investing
in the technology; instead, they already knew
the professor and were investing in him.
But professors are not nearly the only people
who can do research.
> - Allow me to be clear. They are not ignoring e-mail.
No, flatly they are ignoring e-mail. Write them
something short, clear, and astounding, and they
just ignore it, won't read it, won't respond at all.
Maybe they want an introduction. Maybe they want
only other means of selecting entrepreneurs. Maybe
they just are having fun pissing off entrepreneurs.
Maybe they are busy enough with what they are doing
now. Maybe whatever, but they ignore e-mail. Sorry 'bout
that, but they do.
> - You seem to keep missing the obvious point of perhaps they evaluated the technology you seem to be so bunched up about...and didn't care for it.
Nonsense. For the technology in question, there is likely
and apparently only one person in the world who knows it.
Send A16Z a high level view, and they ignore the
communications. They make no attempt to get the
details of the technology and evaluate it. It's
simple: They just will not get involved in evaluating
new technology.
> - Perhaps they evaluated and weren't impressed. Or is that impossible?
It is impossible. They can't have even as much
as a weak little hollow hint of a tiny clue about
anything about the crucial internals of the technology.
They received only a business view of the
power and value of the technology. For how it works,
which is crucial for evaluating it, they don't know.
To know they would have to do some due diligence. There
is little hope that they could competently direct the
due diligence.
> - Fantastic. Again we're not talking about the broader VC industry.
Right. We're talking about A16Z. My point was in
response to your figure of 80%: Likely a VC firm
with ROI just better than 80% of the VC industry is
still losing money for its LPs. In this case,
even if A16Z has better ROI than 80% of VC firms,
they might still be losing money. However, since they just
raised $1.5 billion, I suspect that it appears that
they are making money; since a lot of the old
investments likely have yet to exit, it might be tough
to know their ROI so far, but no doubt they had
to show something good about ROI to raise $1.5
billion.
> Incorrect. What about Oculus for example?
I don't know that case well, but my superficial
understanding is that the product was in production
and selling and all a venture firm had to do
was to evaluate the sales figures and the
product itself without much concern for any
crucial internals of the technology. Like I said,
the US DoD, CIA, etc. funded the SR-71 when
nearly it was was engineering on paper, and VCs
would want to wait until the plane was flying
successfully and then maybe pay for some of the
jet fuel. US IT VCs just will not evaluate technical
work just on paper; they won't do it. Maybe
this situation is enforced by the LPs, but
that is the situation. The story used to be
that VCs could fund a project described on the
back of a napkin, but now they just will not
fund original technical work just on paper or
even when the corresponding software is production
quality and running. If some such project
gets funded, then the reason was not just the
technology and software but the reputation of the
founder or some such. NSF, NIH, DARPA,
Ph.D. committees, and peer-reviewed journals
of original research will evaluate new work
just on paper, but US IT VCs just will not.
And, e-mail with any such connection with
such research goes into the bit bucket,
from their "filter" if they look at the
e-mail at all.
We are in agreement: They just do not want to
look at e-mail.
So be it.
The project in question, if it starts to work at all,
will quickly generate revenue
of $10,000, $40,000, ..., $800,000
a month starting with just a
$1500 server and, for $1 million a month,
using just a few such servers in
a spare bedroom with some A/C in a window,
a consumer Internet connection (that permits
commercial usage), and a 200 A circuit breaker.
So, why jerk chains of VCs? For the project in
question, it has long been the case that
there has been no very good reason,
although maybe just as 'reserve fuel'
in case of some disaster. The main reason
for my posts here are just to let the HN
community know that A16Z and essentially
all of US IT VC just will not take seriously
descriptions of technical work in e-mail
without an introduction, knowing the
person, etc. For anyone with a background
in doing projects inside a business, for
US national security, or for academic
research, this situation is just bizarre
and has cost the founders of the project
a lot of time although only some time ago.
But the project moved forward on its own
funding, and revenue with good free cash
flow, plenty enough to grow capacity
if usage does grow, seems quite near.
So, for this project, to heck with VCs.
But the time wasted in the past on
VCs still hurts. And the HN audience
should know some of the truth.
It sounds like you have everything figured out, the solution is beyond comprehension for mere mortals, and that you'll have 100s of millions of dollars shortly. Perhaps with your new found wealth you can open an IT VC, invest in email filtering, and show everyone here how it should be done. I wish you best of luck in your future endeavors and hope you find success.
> It sounds like you have everything figured out, the solution is beyond comprehension for mere mortals, and that you'll have 100s of millions of dollars shortly.
That's definitely the plan except for the part about
"mere mortals".
Mortals suffice, but not "mere mortals": Instead need
some help, or, say, need to "stand on the shoulders of
giants". The giants in question are not in computer
science -- "a field with much that is new and good however
the new is not good and the good is not new", too close to being fully true.
Without standing on the shoulders of the right giants,
your "mere mortals" is fully correct -- can't do it.
That is, one guy has, in practice, no chance of
reinventing all that stuff needed from the giants. Indeed,
it's tough enough just to get through the exercises
in one of the better books needed. More than one guy? For the original work needed, that is nearly always done
by just one guy; two are less good.
I've already "shown how it's done"
in technology many times for US
national security, in US business,
and in research for US national
security and business.
The US DoD, DARPA, NSF, and NIH
have long shown the world
"how it's done", in total
for 70+ years.
The simplest arithmetic shows that
US information technology (IT)
venture capital (VC)
is not a very good business
compared with consumer Internet:
That is, a US IT VC firm might
get up to assets under management
of $5 billion or so, but the
founders of Google, Facebook, and
more have more than that in their
personal net worth.
One problem with VC is, even if
they decide to take their e-mail
seriously, really the best they
can do is just from what arrives in their
e-mail, and far from VC there are
some serious obstacles here:
First, a standard assumption is that,
since computers are to be used
in the work (they likely do remain
the great largely unexploited
opportunity), the assumption is that
computer science is the best
academic background. Well, that
assumption is false; if want to
be quite serious about the work,
then that assumption is badly false.
Second, for really good results, it will
be necessary for some good people
with good academic backgrounds
in crucial fields and topics outside
of computer science to pursue
entrepreneurship, and so far the
appropriate US academic culture
discourages this.
For this assumption, the US DoD
knows that it is largely false and, thus,
concentrates on pure and applied
mathematics, mathematical and
applied physics, and some
relatively mathematical topics
in electronic and other parts of engineering.
The DoD makes very heavy use of computing
but not so much of computer science.
So, for good results in VC, it will
be necessary to catch up with the
DoD and get good people with the
'right stuff' from academic fields
other than computer science, and
that catch up effort will be difficult.
Really, for good projects, the
situation is much better on the
entrepreneur's side of the table
because there one can cook up
terrific stuff instead of just
waiting for it to arrive via e-mail.
My point that VCs and A16Z ignore
their e-mail really was well supported
by your statement that
e-mail is about the worst way to
contact a VC partner. Okay,
the VC partners don't want to
take e-mail seriously. Bizarre,
but true, and so be it.
For the project in question,
once it gets enough 'traction'
to be of interest to a VC,
the revenue will be way past
when the project would accept
equity funding. E.g., one VC
responded that they wanted to
see 100,000 unique visitors a
month before investing. Okay,
but for the project in question
that would be about $40,000
a month in revenue from
a server with $1500 in parts and
an Internet connection costing
about $100 a month. With
the $40,000 a month, VC
equity funding, a C-corp with
VCs on the Board, etc. would be
about as welcome as a skunk at
a garden party.
One big reason: For the project
in question with its technology
the VCs can't evaluate, there
will be more such work to do,
less important than the work done
already but, still, very much worth
doing. But, of course, the Board
would have to approve the organization
and budgets for such work. Tilt!
A Board of VCs would not be able to
do such work because they would not
be able to understand the project
they were being asked to approve.
So, such a Board would a hole in the
bottom of the boat of the company.
The current A16Z Web site mentions
that the number of VC firms has been
shrinking. Since they refused to
read their e-mail, easy to believe.
Darwin has a lot of work to do and, thus,
is a very busy guy; still Sand Hill Road
and Winter Street are due for
a visit from Charles just any day now.
Likely what will be left are firms
willing to read e-mail and evaluate
projects with new, advanced, powerful,
valuable technology that is the
crucial core of a valuable business.
In the meanwhile, entrepreneurs should
concentrate on some good research for
some good projects that can be
brought to market and significant
revenue with, say, a server from
$1500 in parts and an ordinary
Internet connection. The research
is the crucial part.
Sure, VCs won't invest in projects
based on research, not even if the
research is already in production
quality code. But think of the flip side:
For such a project, VCs won't invest
in any competition, either!
A big part of investment is pattern recognition. Of course they try to understand what people are doing but that only gets you so far. Ideally, you would want to be able to learn from your previous decisions and improve on your process. If you have real data and a way of using it for more insight then you can make much better decisions.
Their version of "pattern recognition" is just hopeless for what they are trying to do.
E.g., how to use pattern recognition in the industry of leading edge military aviation to evaluate what Kelly Johnson said about building an airplane that would go
at Mach 3+ and 80,000+ feet for 2000+ miles without
refueling? Can't do it. Instead have to look at the
mathematics, engineering, etc., especially a special
engine just developed at a special Pratt and Whitney site
in Florida.
Information technology venture capital needs to find
projects that have powerful, valuable new work and
are exceptional. There is no way to evaluate
such projects by simple, empirical
'pattern matching' from the past of
business and venture funded projects.
Venture capital looks at sectors,
the demeanor of the founders,
maybe their socks,
the jut of their jaw,
etc. because that is all they
know to look at. One could count
on two hands all the information
technology venture partners in the
US able to do a competent review of
projects submitted to the NSF.
E.g., the last paper I published
in computer science is for a quite
practical problem in practical computing,
but I doubt that anyone at A16Z
could read that paper or even
direct a competent review of it.
Thankfully for US national security,
how to review projects in technology
is very well known and done very
well everyday, for 70+ years, by NSF, NIH, DARPA, etc.
For Silicon Valley for information
technology (the situation for biomedical technology is
quite
different), there's essentially no chance of
competent technical review, not at A16Z,
KPCB, Sequoia, Benchmark, etc.
For an analogy,
for the SR-71, Lockheed could build several,
have some fly successfully over Russia
and get some fantastic pictures, and
then venture capital might invest
to buy some of the jet fuel. Instead,
the USAF, CIA, etc. did some really
good work reviewing the really excellent
work of Lockheed, all from what
Lockheed submitted just on paper.
Net, Silicon Valley venture capital just absolutely,
positively, flatly doesn't have even as much as
a weak little hollow hint of a tiny
clue about how to do or evaluate original, exceptional,
powerful, valuable
information technology projects. Sorry 'bout that.
"Net, Silicon Valley venture capital just absolutely, positively, flatly doesn't have even as much as a weak little hollow hint of a tiny clue about how to do or evaluate original, exceptional, powerful, valuable information technology projects. Sorry 'bout that."
- Your use of hyperbole is amazing. The original point of venture capital was to "aid in the development of new or existing businesses into companies of stature and importance". Venture capital arose after world war 2 when banks wouldn't bankroll some of the businesses and new ideas that were coming out. Since then it could be argued that VCs direction has changed to investing the 3rd party pooled money into businesses that are employing some novel technology or processes. VCs typically have employees who are familiar with industries, business, and / or technologies to help identify candidate investments as well as to help said investments to flourish. Finally to brush the successes of venture capital over the years in IT projects is just disingenuous. VCs had quite a bit to do with DEC, Fairchild, the whole damned semi conductor business, and the expansion of the computer industry. Was a lot of it on the backs of other projects and companies? Of course. DARPA, Xerox, etc. played HUGE roles in the advancement of technology.
> Since then it could be argued that VCs direction has changed to investing the 3rd party pooled money into businesses that are employing some novel technology or processes.
The information technology (IT) VCs are essentially
hopeless at the crucial work of evaluating the
"novel technology". The US DoD, NSF, NIH, and DARPA
are quite good at such evaluations.
> VCs typically have employees who are familiar with industries, business, and / or technologies to help identify candidate investments as well as to help said investments to flourish.
For good exploitation of technology, what you are
describing is not nearly good enough. The IT VCs
need very much to up their game to whole new
levels. They need, say, to be able to do or
at least direct evaluations of anything
that arrives in their e-mail
in IT that might make a big pot of money,
and they can't and won't do that.
My guess is to have an in-house team that can help out the companies they fund. I know that Netflix's cloud architect, Adrian Cockcroft, left to join a venture capital firm to advise on scaling issues for companies that firm funded[0].
My guess would be a team to provide a data-centric approach to investment. Crunch a whole lot of numbers to identify industry trends and specific firms that are promising, and see if those can replicate human performance.
Or they're simply handling recruitment for a stealth portfolio startup.
Why? The VC firms never invest in an "industry", i.e., there are no such 'index funds'.
Instead, VC firms invest in individual companies, and there
what is just crucial is what the company does. If
the company looks good, then mostly to heck with the
darned industry.
> Or they're simply handling recruitment for a stealth portfolio startup.
In this case, poor startup: A16Z is doing a "piss poor"
job 'helping' the startup!
Does this mean Mattermark https://mattermark.com/ have started a tech arms war amongst the larger VC firms to create their own toolsets to differentiate their deal flow from others?
Mattermark has gone some way to provide proof that data can make a difference, at least at highlighting opportunities that might not have been visible before. I wondered whether this would result in herding investors towards some companies, which means differentation and opportunity comes from spotting things the others miss... finding the outliers that Mattermark miss.
Interesting that the test is "make a dashboard", well into the standard skillset for most analyst programmers who work for financial institutions (I built a lot of this stuff for the finance sector in London).
The overwhelmingly largest share of returns accrue to capital. It's a good idea to resist being cast in the role of simply "programmer" over time, for this reason.
I love that people do this. Do you really thing someone is going to write a web scraper and NOT include obvious things like converting AT -> @ and DOT com -> .com?
Also, who even worries about spam any more? I haven't seen actual spam in my inbox in years.
My problem is I like taking these tests to find out what I don't know (quite a lot usually). To be fair though code reviews usually tell me more than tests these days.
The constraint that gets me is the honor code - I cannot use an IDE with code completion?! but that leaves nano or notepad ...
The test's restriction on consulting material from websites and books, excluding the official reference documentation boggles my mind. Unofficial 3rd party documentation is standard kit for a developer, especially when using community driven programming languages.
And... my suspicions were confirmed. I still have no idea how I did on the test and they have provided no feedback.
So, unless something changes, which I don't predict, it was a waste of time I wouldn't recommend doing the test nor applying to work at a company like this.
Have you guys thought about possibly employing smarterer.com rankings? I took your test, I think smarterer may yield a better evaluation, although you can't write code on a clock.
This is a very difficult question and, thus, illustrates
the transcendent brilliance and astoundingly
"deep domain knowledge" of A16Z. An answer is
available only on the bleeding edge of the
most brilliant insights of research in
machine learning. Ordinary people can't expect
to understand such depth! All bow down to
the incandescent brilliance of A16Z -- after all,
one of their founders used C++ to write
a GUI application to read a text markup
language and display the formatted results!
Maybe this was the first time, I mean after
Word, TeX, and some others?
A BS/MS/PhD in Computer Science or the equivalent is nice
to have, but the most important requirement is significant
independent programming experience as demonstrated by your
GitHub account, personal projects, academic publications,
or startup success. Your accomplishments are much more
important than your paper credentials.
Who says they're not recruiting other kinds of people through other channels? This kind of test is appealing to some people, and I'd guess those people have a github account.
Looks to me like A16Z is blowing it. If Ben Horowitz
prides himself on good management of technical work as suggested by his blog posts, then he didn't work very hard on the project of the OP.
Issues:
(1) Engineering School. Uh, guys, here in the US we have
universities with schools of engineering -- MIT, Johns Hopkins, CMU, Georgia Tech, etc. Running a good engineering school is a lot of hard work. But this A16Z project has A16Z being their own engineering school, that is, specifying 'course material', testing for it, and deciding on pass/fail. Somehow it seems a bit arrogant for
A16Z to believe that they can do the work of an engineering
school.
A16Z gets really off the track with their,
> A BS/MS/PhD in Computer Science or the equivalent is nice to have, but the most important requirement is significant independent programming experience as demonstrated by your GitHub account, personal projects, academic publications, or startup success.
Wow! "PhD in Computer Science" ... "is nice to have, but"...! Gee, guys, just shut down MIT, Johns Hopkins, CMU, Georgia Tech, etc. because now A16Z is here with "the right stuff"!
Next, for their "as demonstrated by ... academic publications", sorry, A16Z, "academic publications" are not
at all about "programming experience". Instead, the usual criteria are "new, correct, and significant" where "programming" really does not count more than, say, just the word processing for the paper.
(2) Machine Learning. Good grief! That "machine learning" is about "scatterplots & histograms, logistic/linear regression, R/Matlab" fills a "much needed gap in the literature" and "would be illuminating if ignited". Their definition is for a 'statistical data assistant' mostly just for 'descriptive statistics'. Ever hear of J. Tukey? Ah, I'll be nice to you guys; not fun to be too harsh on small children!
Then there is their unforgettable,
> If you also know enough data science/machine learning to design useful metrics for dashboards and workflows, that’s a major plus.
Uh, ever hear of reliability and validity? You have three
hours to respond here with a relevant description! You are not permitted to use Google or Wikipedia!
(3) Ego. It appears that the A16Z specification was
written by someone who has enough ego to believe that
their own 'skill set' is just "the right stuff".
(4) Linux. People programming on Microsoft's
platform can have done well but encountered
only a small fraction of the topics in the A16Z
document. The document is mostly about Linux.
Sorry, guys, I'm writing in Visual Basic .NET;
no, C# has too much of the deliberately 'idiosyncratic'
syntax of C which I find too error prone and hate.
(5) Three Letter Acronyms (TLAs). It appears that A16Z
is using obscure TLAs as code word filters. Hmm.
So, they have ORMs. Gee, what the heck is that? So, Google a little and get a Wikipedia entry and discover ORM abbreviates 'object-relational mapping'. Okay. Sure, if write the software with objects and keep the data in a relational database, then need move data between objects and tables. Given the huge variety of objects and tables, tough to believe that there is a significant 'subject' here.
Then there is their ETL. Google again and discover that
that TLA abbreviates "extract, transform, and load". So that is essentially 'extracting' data from whatever
original sources, 'transforming' the data as needed by the work to be done, and 'loading' the data into a data base. Okay. Sounds entirely routine. I'm surprised they didn't also list (A) LRT, look at a screen, read it, and type in what need, (B) PSS for put on shoes and socks, (C) SDP for start car, drive to the office, and park the car, and
especially the crucially important (D) GSL for Google a question, select the promising results, and learn what needed. One more: WTH, waste time on Hacker News!
Okay, A16Z, two can play this game: Think of a large server farm or internet protocol network. Since this 'facility' is subject to errors of wide variety, it is important to monitor the facility for unexpected problems of wide variety, say, from software errors, hardware failures, system administration errors, security breaches, etc. To do this collect some data, say, from Microsoft's instrumentation means, HP's Mercury Interactive, what a relational database management system or Cisco box can report, etc. So, maybe 100 times a second, collect numerical data on each of 25 variables. Now we want
a detector of 'anomalies', including for problems never seen
before, that is, 'zero-day' problems. Such monitoring is
essentially hypothesis testing. So, we want a suitable
hypothesis test. Due to the fact that the multi-dimensional
probability distribution of that 25 dimensional data
is essentially impossible to describe at all accurately,
we need an hypothesis test that is 'distribution-free'.
Since the data is 25 dimensional, our test must also
be 'multi-dimensional'. We want false alarm rate
known in advance and adjustable in small steps over
a wide range. And we want a good shot at doing nearly
as well as the Neyman-Pearson result but with much
less data than that result requires. Okay, A16Z, your
mission, and I'd like to see you accept it, is to
say how to do that. Of course, since you guys are so
good and brilliant, you already know my published paper
for the answer. But there is a typo in the paper. Where
is it? You have three hours to post your response here.
Also needed, find a fast way to do the specified
computations. I found a way, but that is not
in the paper. You have 24 hours to post the
answer to this part! Don't report your answer
in some silly way such a running code. Instead,
report your answer in clear English
with any appropriate mathematics written
as high quality mathematical writing,
e.g., W. Rudin. Go for it, guys!
Uh, you
won't find the answer on Google or GitHub!
And asking around Silicon Valley won't help, either.
You might want to rush off to the
RAD lab project of Patterson and Fox funded
by Google, Microsoft, and Sun for when
they worked on such problems, but that won't
help, either.
My view: Here A16Z showed that their understanding of
computing is still down at the level of a junior high
self taught programmer.
(Incidentally, PG designed the /jobs posts to be the one privilege that YC startups get on HN. Other than that, the site policy is to treat them the same way as everybody else.)
It's not uncommon for posts to be in a grey area between job ad and story. A programming test for an interview process doesn't seem all that grey, though, so I've been killing these. Sorry to be a hard-ass about this, but I don't want to be accused of enforcing the policy erratically.
Edit: I'll unkill the post (but otherwise leave it demoted) so the discussion can remain open.
Edit 2: The argument has been put to me that this is newsworthy because A16Z are doing it. Since that's corroborated by external sources (e.g. Twitter), and the post has gotten a lot of upvotes even while it was buried, that's reasonable—so we'll restore it as news rather than as an ad. Besides which, HN is one of the few places where a programming test counts as interesting.