True, OSS is also usually more philanthropic in that many times it isn't directly a paid situation or most times which changes the dynamic. However many people do OSS to attract business with as well or specifically to do business on that isn't controlled by a company i.e. Linux, Rails, Python etc.
I am sure in a flat company, compensation is probably the toughest part as it can easily turn into a popularity/tribal thing possibly unless there is some set ranges and agreed upon structure. I wonder in a flat company if compensation needs to be more open.
The difficulty lies in quantifying experience - there loads of ways to pull that off (and none of them quite wrong). Years of experience, type of hackery, FOSS contributions, etc. I'd say the best option would be to pick one and stick with it. If you're properly open about it, those who take issue with your system won't apply.
I am sure in a flat company, compensation is probably the toughest part as it can easily turn into a popularity/tribal thing possibly unless there is some set ranges and agreed upon structure. I wonder in a flat company if compensation needs to be more open.