You're right, a proper extension of Shannon's entropy to the continuous case requires a reference measure. Jaynes discusses this in the Brandeis lectures I linked to in another comment: https://bayes.wustl.edu/etj/articles/brandeis.pdf
I see, of course in physics you have the advantage that the phase space already has a canonical volume form, which coincides with the normal uniform measure if you use canonical coordinates and, amazingly, is preserved by the equations of motion.
In most statistical problems you don't have such a nice measure, and it is always good to keep in mind that choosing a uniform 'improper prior', even implicitly, will mean your choice of coordinates will influence your result.
Edit: Hang on, I just noticed the names on the lecture notes you sent me. Uhlenbeck, Wheeler and Schwinger? That's one heck of a line-up. With the part you linked by Uhlenbeck it seems, I'm going to set aside some time to read that one more carefully.
The pdf I linked to contains only pages 181-218 from Vol III of the 1962 lectures (Jaynes). I’ve not read the others, I don’t know if they may be available online.
