I suspect that so many do it because it's so often repeated and taught that it works; but they figured out it works when looking at the whole of youtube videos, which percentage wise, is mostly targeted to a younger, less mature audience, and the content is made to match. When a channel's target audience is a similar demographic to the "average youtube viewer," it works very well. But I'd wager it doesn't work very well when the target audience doesn't match the average youtube viewer. I'd love to know if the people that measured and promote these bright, astonished-face, click-bait thumbnails ever bothered to test their effectiveness with different target demographics. My guess would be no.
I really wish the people making videos on subjects like science/physics, engineering, woodworking, etc... would stop putting thumbnails and titles on their work that target an audience that isn't going to care/enjoy/understand the subject of the video. And maybe even keep some of the intended audience away.
> I'd love to know if the people that measured and promote these bright, astonished-face, click-bait thumbnails ever bothered to test their effectiveness with different target demographics
They have bothered to test the effectiveness, and found that clickbait does work. (You'll sometimes see a video title change over time as creators try to find a title that drives more engagement.) The reality is that whatever gets more views or engagement is more valuable to creators -- target audience really doesn't matter.
In this world, most of the money you get is from sponsorships and YouTube ad share, which requires raw views. (Something the "intended audience" doesn't care about!) For a creator trying to make money, the "intended audience" is whoever watches your videos. Does it matter to the creator if the "intended audience" is kept away when the creator ultimately makes more money by boosting views through clickbait?
The video I linked specifically refutes much of your speculation. LTT included their data showing that the clickbait thumbnails improved viewership among their older-skewed audience.