I think if frame it as, “each 100ms you shave off a critical consumer shopping checkout flow page increases conversions by X%” it starts to seem more defensible.
Agreed that in the general case 100ms to full paint (with interactivity not far behind) is really good.
Doesn't really matter if the relationship isn't linear like that. I suspect it's more likely conversions start to drop off after certain thresholds are met e.g. 3 seconds causes conversions to drop by 5%, 20 seconds causes them to drop by 30%.
Just averaging out those numbers could result in engineering time being wasted chasing rapidly diminishing returns, if the site is already below one of those thresholds.
Agreed that in the general case 100ms to full paint (with interactivity not far behind) is really good.