Yeah, I think it could do with a modal interface: normal mode would perform tree-like operations and allow entering text. Insert mode would treat the entire file as plain text.
I meant modal as in "if you hold down backspace, the item's text will be removed, but the item heading will remain", i.e. the headers are read-only while editing an item. evil-org-mode won't stop me pressing backspace too many times and screwing up.
You could use hydra [1] to set up something like this pretty easily. I did have a setup like this once, but found that escaping the 'modes' with an extra keypress when I only typically wanted to do one or two operations wasn't particularly nice.
... there must be a mode that does this already?