"Just because I can write `f(x)` as `f x` with full featured operator precedence and associativity tables doesn't mean I can introduce new syntax into the language."
There's two definitions of "DSL" and I find mixing them to be a bad idea. I prefer to confine "DSL" to a fully-fledged actual language, with its own parser and evaluator. Haskell is just about the only language I know that makes this easy enough to actually consider as the solution to a problem.
Then there's "glorified API to look like a language", a definition I don't like but is clearly in current use, and the one used in this post. Haskell is pretty decent at that, but of course you can't escape from the fact you're using Haskell. But then, people pretty freely use the term "DSL" in Ruby where you can't escape the fact that you're in Ruby. In either case, "adding new syntax" is not actually part of the general definitions of DSL nowadays. The "adding of new syntax" is an illusion, and I guess that's part of why I don't like this definition of DSL, there's actually an element of deception in it and in practice I have found people not 100% comfortable with the base language the DSL is in often end up deceived and less effective. And either way, if you really, truly want some new syntax, Template Haskell can mostly keep up with Lisp anyhow, but not entirely. There may not be, at this moment, a precise match for the rx package already on Hackage but I believe Haskell has all the necessary functionality to create one, up to and including compile-time regular expression compilation. Though given the way Haskell works, it is also the case that compile time RE compilation is way less useful, because it's easy to make it compile only once even at run time with the way Haskell works. As is so frequently the case, what takes a macro in Lisp does not require a macro in Haskell.
"This is either weird nonsense or the old variable capture argument."
No, it's a functional purity argument. Lisp in practice has the same problems with mutation and state as pretty much any other language. Insert debate about importance of functional purity here. (That is, I'm not making the argument now, I'm just letting you know that's what was being referenced.)
That's a subset of the functional purity problem. It's a relatively-commonly-understood one, but Haskell has taken the argument far further than most, such that that is now only a subset of a larger argument about state and mutation. The argument is not merely that you've got some state being mutated in a bad way, the argument is that the fact you've got state mutating at all is a bad thing. In this case, it's state in the compiler phase, but it's ultimately the same argument. The argument would be that combining any two things that mutate state is inferior to trying to combine two things that don't, at any level.
I'm personally still skeptical of this general argument, but I mean that in the "true" sense of skeptical. I think in a lot of ways the benefits have panned out as promised but there's some costs still swept under the rug. Rather a lot of the development ongoing even now in the core Haskell community can be viewed as various ways of lowering or eliminating the costs associated with their approach.
(For instance, the still-ongoing, but rapidly converging, changes associated with trying to create a decent string library that does not treat strings as linked lists of integers. On the one hand you can see it as normal feature work, but on the other it's a way of trying to go from having an unusually bad string story induced by functional purity to potentially having an exceptionally good one. It's this sort of work that actually keeps me interested in Haskell; I don't know of any other community doing so many actually new things to prove out their philosophy, instead of rehashing mutation-based OO in yet another way.)
There's two definitions of "DSL" and I find mixing them to be a bad idea. I prefer to confine "DSL" to a fully-fledged actual language, with its own parser and evaluator. Haskell is just about the only language I know that makes this easy enough to actually consider as the solution to a problem.
Then there's "glorified API to look like a language", a definition I don't like but is clearly in current use, and the one used in this post. Haskell is pretty decent at that, but of course you can't escape from the fact you're using Haskell. But then, people pretty freely use the term "DSL" in Ruby where you can't escape the fact that you're in Ruby. In either case, "adding new syntax" is not actually part of the general definitions of DSL nowadays. The "adding of new syntax" is an illusion, and I guess that's part of why I don't like this definition of DSL, there's actually an element of deception in it and in practice I have found people not 100% comfortable with the base language the DSL is in often end up deceived and less effective. And either way, if you really, truly want some new syntax, Template Haskell can mostly keep up with Lisp anyhow, but not entirely. There may not be, at this moment, a precise match for the rx package already on Hackage but I believe Haskell has all the necessary functionality to create one, up to and including compile-time regular expression compilation. Though given the way Haskell works, it is also the case that compile time RE compilation is way less useful, because it's easy to make it compile only once even at run time with the way Haskell works. As is so frequently the case, what takes a macro in Lisp does not require a macro in Haskell.
"This is either weird nonsense or the old variable capture argument."
No, it's a functional purity argument. Lisp in practice has the same problems with mutation and state as pretty much any other language. Insert debate about importance of functional purity here. (That is, I'm not making the argument now, I'm just letting you know that's what was being referenced.)