One thing that pops out there though is the underrating of non-Western universities. Hungary has a history of producing scientific / intellectual talent, and it appears that Szeged is ranked #3 in Hungary, yet it's 712th globally. It would seem that the ranking list isn't very reflective of intellectual talent.
For an even more glaring example, even the best of the incredibly selective IITs don't make it into the top 500 in the US News global ranking list.
The thing is many of these rankings put a great deal of emphasis on research program quality and impact. For IITs at least, the research programs are nowhere as good as many top western universities.
I studied undergrad in an IIT faculty that's considered the top in its field in India by reputation of the professors and I worked on two research projects with PhD candidates there. I also spent one summer working on a research project at KTH in Sweden. I must say the experience in terms of research quality, capabilities of the PhD students, facilities, support systems etc was night and day. Due to this in the IITs, the top undergrads leave the country to pursue post-grad and the folks left behind are often the ones who didn't get admits with financial support outside. Very often I observed the undergrad students assisting with the research understood things better than the PhD candidates themselves and all these makes a difference in the research quality and impact.
Now, if you rank university programs in terms of % change in life circumstances, IITs will very likely come at the very top. Because the entry is entirely meritocratic with minimal reservations and even reservations are held to academic standards - seats go vacant rather than being filled with less qualified students - a lot of students from poor families get boosted solidly into upper middle class with an IIT education including many of my own batch mates.
Rankings like ARWU or US News are quite biased towards big research universities.
Nature and others have released rankings that are a bit normalized by size, and many niche institutions such as Cold Spring Harbor or the Austrian IST are near the top. This was discussed in an old HN post, which is quite interesting to read through: https://news.ycombinator.com/item?id=20231432.
ARWU subject ranking is pretty good, though. Rankings that are not subject based are hard to interpret. For example, many smaller Ivies have excellent departments in topic X but are not so good in topic Y. They can't cover everything. But if you trust the ranking and go there anyway to study topic Y, you might be disappointed.
However, funding is really biased towards big or famous institutions. In my experience, it's really easy to get tons of grants from a place like Oxford or Cambridge. If you move to an excellent place that is less known, say a university in the top 30-50 of the world, it gets a lot harder even if your proposal is identical.
There's also a big implicit bias towards the concept that research is done in universities - in some places there's a distinction of institutions where universities focus on teaching and most of academic research is done outside of universities in separate research institutes (e.g. the Fraunhofer institute chain in Germany which employs many thousands of researchers and has resulted in a lot of high-impact science); a professor might be involved in both, but his research work wouldn't count for the university's rating (because it's not affiliated with the university) and the institutes are excluded from that rating because they don't teach undergrads.
Rankings can also be affected depending on how the identification of the university in the affiliation is done. Automated matches are often lower for languages that have different ways of writing things (often when there are different characters, or it's written in different languages), changes to location/name/structure, and how long it's been around. Based on a quick look at wikipedia:
For Szeged, the university is kind of old but also only appeared in it's current form in 2000.
It's also called Szegedi Tudományegyetem.
It has multiple colleges inside it, so it may depend on how well those are represented in the data.
Authors using acronyms more makes identification less likely, each of the colleges have acronyms.
Finally, where authors publish makes a big difference - not all publishers open their data up and even those that allow access to some raw data it can be less reliable to extract. So if you publish in places where the publisher does not make the affiliations easily machine readable your work may be counted less in many of these automated rankings. Pubmed (not a publisher but a key way a lot of people get some of this data) didn't have affiliations for anyone but the first author until some fairly late date.
Either automated mappings are used for a lot of work (since there are tens of millions of unique affiliation strings), or the source material is heavily curated (so for example only the "top" N journals are looked at).
source: worked on affiliation mapping at Dimensions in a previous life
That's true. Some rankings like the one from Nature I mentioned do include institutes. As a result, places like Wellcome Sanger in Cambridge or Helmholtz in Germany rank really well and above most leading universities in their field. As you say, these are not universities, but research institutes and therefore ignored by ARWU, US News and others.
The US News ranking is known to be totally garbage, I really wish people would stop citing it. csrankings.org, which tries to be more objective, has Szeged tied with a number of other schools for at 432 globally (of course that's only considering the CS department).
(FWIW I think ranking schools like this is generally harmful, and even "objective" rankings will still have plenty of biases baked in)
Hungary is an interesting example because it's been pointed to in recent years as adopting the "funding the person" model and trying out different funding approaches.
The broader issue you're mentioning is spot on, though: what are we incentivizing, really?
Things are so broken in academics, at least in the US. I'm not sure what is better but the current system just isn't working.
For an even more glaring example, even the best of the incredibly selective IITs don't make it into the top 500 in the US News global ranking list.