There is some virtue to your criticism, but it is also not entirely true. There are things you can check to see if a team is doing Agile for real, like, what's the last process change you experimented with to see if it improved your outcomes? What experiment failed and was dispassionately removed from the process because it didn't work out? When your team's workload nature changed (e.g., from new development to maintenance), how did you adapt your processes to match? Do your processes come up from the team and their experience or do they source from above with effectively no feedback?
At that point, if you are doing those things, there is a certain amount of validity to the criticism that if it didn't work, you really weren't doing it right. Either something external jammed you up so you weren't able to adapt and truly follow agile, your team was personally unable to execute on the adaptation for some reason (structure, personality issues, experience levels, being simply too large to be able to be this flexible because large teams simply need more structure), or the task was simply too hard in the first place for some reason (such as "it doesn't matter how agile and smart the team is, you're not getting 4 people to produce a standards-compliant browser that is also an office suite in six months").
At that point, if you are doing those things, there is a certain amount of validity to the criticism that if it didn't work, you really weren't doing it right. Either something external jammed you up so you weren't able to adapt and truly follow agile, your team was personally unable to execute on the adaptation for some reason (structure, personality issues, experience levels, being simply too large to be able to be this flexible because large teams simply need more structure), or the task was simply too hard in the first place for some reason (such as "it doesn't matter how agile and smart the team is, you're not getting 4 people to produce a standards-compliant browser that is also an office suite in six months").