My guess about the use of lua would be simply because he don't want to commit it to pandoc yet. If you push it to pandoc, it's next to impossible to remove it, or change its behavior in a backward incompatible way.
So naturally, since the embedded lua is expanding its functionality over the years, including the ability to write custom writer and reader, this is the way to be able to tap into pandoc without committing. Tapping into pandoc is important here, as the community who likely will provide feedbacks and insights are there.
I'll bet eventually when it becomes mature, a native Haskell implementation will comes.
So naturally, since the embedded lua is expanding its functionality over the years, including the ability to write custom writer and reader, this is the way to be able to tap into pandoc without committing. Tapping into pandoc is important here, as the community who likely will provide feedbacks and insights are there.
I'll bet eventually when it becomes mature, a native Haskell implementation will comes.