Python uses text mode by default to avoid surprising beginners on Windows. If you only use Unix-like OSs you will never have this problem.
Opening a file should return an object that gives you bytes, and that's it.
This "mode" thing is idiotic, and leak a low level API that makes no sense in a high level language with a strong abstraction for text like Python.
Text should decoded from a wrapping object. See my ohter comments.
Python uses text mode by default to avoid surprising beginners on Windows. If you only use Unix-like OSs you will never have this problem.