When React was new and compilers/tooling wasn't that good it was pretty common to use it without JSX. JSX is pervasive because the tooling is now first class (and you'll need it if you make a non-trivial app sooner or later...at which point might as well use JSX since its there), but it works perfectly find without. My first couple commercial apps for React didn't use JSX and it was quite usable. There's just a lack of examples on how to do it these days because no one bothers.
The only time when you won't really have access to a tool chain is for quick demos with just an html page and a script tag (if you don't want to pull in babel as a script), but most online tool allow jsx too now anyway.
The only time when you won't really have access to a tool chain is for quick demos with just an html page and a script tag (if you don't want to pull in babel as a script), but most online tool allow jsx too now anyway.