I agree, and would add that this is exactly what the OP is saying when it talks about this being a communication issue. Saying "a branch is just a ref" is the right attitude for talking to git and making it do what you want it to do, but it's the wrong attitude when you are writing or talking about code, in which case branches are the reification of the processes by which your code evolves.
I think most people come to git with their own mental model of what a branch is, what a merge is, etc.
Learning git is often mostly undoing their preconceptions (ie by saying 'a branch is just a ref').
But ultimately, we humans tend to think of a branch as a branch, not a ref. For instance, the previous sentence probably made perfect sense to you.