We are having this conversation assuming a society which rejects slavery in principle. Given that rejection, I don't see how past history of enslaving people is relevant to questions of the future–unless one supposes that rejection is going to be reversed at some point, which seems unlikely.