I am not a neuroscientist, but believe the following to be mainly noncontroversial:
When you speak to another person, there is a neural activation pattern distributed over space and time which encodes, somehow, a sentiment to be expressed. There is also a multistage neural structure (L) that converts that pattern into another, more sequential pattern that drives motor neurons to articulate a series of phonemes.
Sometimes an activation pattern will drive this sort of conversion to yield a serial, phonetic activation pattern, but activity elsewhere in the brain that interferes with the final stage of structure L to prevent the actual articulation from happening. You could call this "covert speech".
You probably perform covert speech at least occasionally. If you stub your toe, you may have the sense that you have just sworn/cursed, even if there was no utterance. (This might be because the processing in earlier stages of L leads to a brain state encoding an expectation of hearing the utterance, but this is getting a bit speculative.) I'll call this sort experience "hearing your own covert speech".
It may be the case that certain types of cognition, like multistep logical deduction, are performed by use of L. This is a small minority of human mental activity, but it is probably the mental activity that we are most aware of performing.
If people go through the day being totally unaware of most of their cognitive activity, but "hear" their own covert logical reasoning whenever they do it, I can see how they would form the impression that they "think in English".
For me, it's not much different than reading silently, where I convert text into mental auditory speech and the corresponding concepts are simultaneously activated.
Once having studied, for example, rational arithmetic, or predicate logic, it is difficult NOT to think in those languages. Yet we are perfectly capable of performing rational or predicative inference if we have no knowledge of those languages, if less precisely. Presumptively this is due to the abstraction of rules from experience facilitating a branching sequence model for abstractions and hypotheses represented during inarticulate thought - abstractions being holographic summary statistics of concepts derived more directly from the sensory manifold.
