A lot of this is stuff I've been thinking implicitly for a while.
Success can exist on a continuum between competitive and niche.
The far end of "competitive" is being something like a concert violinist. Everyone knows that it's prestigious to be a concert violinist, millions of people want to, everyone plays the same audition pieces; you win if you're a hair's-breadth better than a million people doing the same thing. It's an unpleasant process, and the odds of success are long, but you know for certain what you'll have to do to win, and you know that if you get that coveted orchestra spot, you'll have reached the top of your profession.
The far end of "niche" is probably starting a company in a completely open industry. Nobody else is doing what you're doing; you're not competing directly with anyone; but you don't know what you'd have to do to succeed, you don't know if the niche you've chosen is any good, and you don't know how to judge how valuable it is to succeed in your niche. On the other hand, you don't have the demoralizing stress of trying to do the same thing as everybody else, infinitesimally better.
I've been trying to decide what to do with my life in this context. I'm a grad student in mathematics, with research interests related to machine learning. On the one end of the scale there's academia (which is at the extreme "competitive" side of the axis) and on the other side there's entrepreneurship (at the extreme "niche" side of the axis) and in between there's working as a data scientist at companies ranging from "three-month-old start-up" to "IBM." There's a tradeoff between the freedom of staking out your own territory, and the risk that if you're free to make things up, you'll get everything wrong.
(I've noticed that this is also true within science. In my computational vision research, there's a lot of "whitespace" for developing your own mathematical models; but most people, sooner or later, get proven wrong. In my harmonic analysis research there's a lot of classical theory already existing and the problems are better-defined; but it's correspondingly harder to make an original contribution.)
Success can exist on a continuum between competitive and niche.
The far end of "competitive" is being something like a concert violinist. Everyone knows that it's prestigious to be a concert violinist, millions of people want to, everyone plays the same audition pieces; you win if you're a hair's-breadth better than a million people doing the same thing. It's an unpleasant process, and the odds of success are long, but you know for certain what you'll have to do to win, and you know that if you get that coveted orchestra spot, you'll have reached the top of your profession.
The far end of "niche" is probably starting a company in a completely open industry. Nobody else is doing what you're doing; you're not competing directly with anyone; but you don't know what you'd have to do to succeed, you don't know if the niche you've chosen is any good, and you don't know how to judge how valuable it is to succeed in your niche. On the other hand, you don't have the demoralizing stress of trying to do the same thing as everybody else, infinitesimally better.
I've been trying to decide what to do with my life in this context. I'm a grad student in mathematics, with research interests related to machine learning. On the one end of the scale there's academia (which is at the extreme "competitive" side of the axis) and on the other side there's entrepreneurship (at the extreme "niche" side of the axis) and in between there's working as a data scientist at companies ranging from "three-month-old start-up" to "IBM." There's a tradeoff between the freedom of staking out your own territory, and the risk that if you're free to make things up, you'll get everything wrong.
(I've noticed that this is also true within science. In my computational vision research, there's a lot of "whitespace" for developing your own mathematical models; but most people, sooner or later, get proven wrong. In my harmonic analysis research there's a lot of classical theory already existing and the problems are better-defined; but it's correspondingly harder to make an original contribution.)