Preference is mostly social. A good recommendation system would need to feel more like a friend than an algorithm. Not only does it need to understand your preferences, it needs to convince you that it understands "where you're coming from" in general so you can relax and be honest with it. And even if that can be achieved technology-wise, how could you comfortably accept an AI system as your friend if its being "controlled" on the back end by a whole organization of people you don't know? There's more than just tech missing from this equation.
