What are you going to run on it? 99% of userland programs expect compatibility with some historical version of X11. Are you going to rewrite GNOME and KDE and everything else?
Many userland applications don't use X libraries directly, but use a toolkit like Qt and GTK. Both projects are working on Wayland support [1, 2], bypassing X completely.