It is pure in the same way that Rust is memory safe. That is too say there are a tiny number of exceptions/escape hatches, but they are not meant to be the norm. Every day programming doesn't involve them.
Exceptions define an effect. Code with exceptions aren't actually pure in the sense that their return type doesn't fully describe what the code does, so it doesn't just map an input into an output: there is something else going on
In some pure functional languages, the pure fragment doesn't have exceptions, and you add exceptions as an effect (or as a monad)
(If you reify the effect with a type like Either or Result, then the code becomes pure: but that's just a monad like said above)
It's fine if the poster above me wishes to change languages. Anyone who chooses to stick with Haskell should get comfortable with the notion that functions can diverge, and accept that this is fully consistent with the Haskell type system. The return types may seem unhelpful, but they are accurate.
Anyway I really like the take that Haskell is pure in the same sense that Rust is safe
Haskell has impure constructs but sets you up to define referentially transparent abstractions with them, and this is actually valued by the Haskell community
Exceptions aren't impure anyway.