The problem is the revenue model. Zombie viewing is more profitable than surprising new content.
If you were asked to hack on YouTube's algorithm, any modifications you'd produce, no matter their merit, would be tested against the status quo for their ability to generate revenue.
I'm sure that is what they tell themselves at youtube HQ. But I'm not convinced. Why should one A.I. fit everyone? It just seems like laziness, ie. corporate inertia, to me. If they could open up an API for third party recommenders this would really be interesting.
Not really. Surprising new content has more active engagement which is crucial for acquisition ads. Revenue for pay-per-click is always better than display.
If you were asked to hack on YouTube's algorithm, any modifications you'd produce, no matter their merit, would be tested against the status quo for their ability to generate revenue.