I would argue that the impact of these "buy and hold" investors on share price is minimal. There's so much liquidity these days that any one class of investor should have a minimal impact on the share price. The index funds end up moving WITH the market, so any downward (or upward) trend will be followed by whatever balancing algorithm they use.
Your big swings in share price are still going to be driven by HFT algo traders, activist investors, hedge funds, etc.
I do agree that companies in the S&P 500 have an oversized advantage in raising capital, but if you remove index funds from the picture, I think the mere existence of the index would have a similar effect (this could be researched historically, but I don't have the time/interest to do so). On the flip side, any stock in the S&P is going to have limited arbitrage opportunity due to the level of liquidity they get from index fund rebalancing, etc.; so algo traders looking at specific plays may want to find an asset with lower liquidity and fewer eyes watching it. As with anything in finance, there is a play available on both sides of the field and they tend to balance each other out.
HFT affect and take advantage of minor price swings[2], and their arbitrage is based on this. Long-term investments are the primary driver of (hence) long-term price movements. After all, HFT, by definition, have no long term interests in issues, so they are (usually) equally weighted in either (long or short) direction, thus cancelling their long-term effects out, even in a situation where there are no long-term investors.
1. This pertains your comment regarding HFT. Activist investors, which you also mentioned, are just long-term investors with a marketing strategy (put your money into something and then go tell everyone), and in some cases a lobbying strategy (e.g., Bill Ackman's notable short of HLF), and they have no place being compared with HFT.
2. It is true that, sometimes, minor price swings can cause a crossing of a tipping point (e.g., below a prior price support level, etc.), but even in these cases the long-term price will adjust for these swings based on intrinsic value.
I was more referring to the liquidity provided by HFTs -- they do tend to speed up price corrections, even if they're not the primary driver. I was referring more to the tipping point scenario -- HFT ensures that asset bubbles don't persist for long.
Good point. Actually there is a thriving segment of the broker and hedge fund market whose entire game is figuring out which stocks will have their weights changed at each new index rebalancing, and "front-running" the index funds who can't do anything until it becomes official. Thus they profit off what is a market inefficiency, scraping value from the passive guys, which in the long run could I suppose limit the index fund phenomenon (though not the indices themselves).
Essentially, if the "front-runners" (not really a correct term, since nothing illegal is going on) weren't doing this, then the index funds would have to pay a huge premium to buy large amounts of stock on extremely short notice. The "front-runners" are actually reducing the index fund's overhead costs (and, yes, being compensated by the market for doing it). They certainly aren't cheating the index fund investors out of anything, because their investments will track the market, regardless.
The "front-running" we're talking about isn't really a huge problem. It's a relatively limited arbitrage opportunity, and the ETF fund runners can just start performing this sort of arbitrage themselves if it becomes too big of a problem. I'm guessing insurance against this kind of activity is already priced into the fees of most ETFs (so yeah, they're scraping value, but not a lot and there's not a lot of opportunity for expanding it).
They absolutely should be doing that, but this market inefficiency is strangely persistent. In the bond world at least, I have watched hundreds of billions under management by sleepy fund managers in Europe who operate like bureaucrats for years: all they want is to be fed the index rebalancing weights each month and they don't take much notice of the value being left on the table by not anticipating changes efficiently. Perhaps its because the industry is biased towards fees rather than performance that this persists, but I don't have a great explanation for it.
I know for sure though that there are fast-money guys who obsess daily about weighting momentum and turning points and they do well quite consistently.
Your big swings in share price are still going to be driven by HFT algo traders, activist investors, hedge funds, etc.
I do agree that companies in the S&P 500 have an oversized advantage in raising capital, but if you remove index funds from the picture, I think the mere existence of the index would have a similar effect (this could be researched historically, but I don't have the time/interest to do so). On the flip side, any stock in the S&P is going to have limited arbitrage opportunity due to the level of liquidity they get from index fund rebalancing, etc.; so algo traders looking at specific plays may want to find an asset with lower liquidity and fewer eyes watching it. As with anything in finance, there is a play available on both sides of the field and they tend to balance each other out.