Child labor is much the same - a society can only ban it when it can afford to do so. An agrarian society has inevirably plenty of child labor, as part of the family. That said, you can probably ban companies from ever employing children, especially with the productivity level of current mass-production tech.
Investment in people is a red herring. You can give people choice and invest in them. Though guess what: developing countries (which is who we're talking about) don't have money to do the latter.
If the reason is ethical than why would we support sweatshop labor, or any other exploitative labor?
What exploited people need is not sweatshop jobs but access to capital and education. Invest in them, do not exploit them.