The universal nature of mathematical truth isn’t dependent on agreement. Instead it’s only universally true when you include all those seemingly unspoken “agreements.”
I'm not sure what you mean by this, particularly this phrasing: "sets of axioms where 1 = 1". 1 = 1 is an axiom, not something that's proven by any set of axioms.
But I'm willing to believe I'm simply naive here. You say there is some set of axioms which prove that 1 = 1. Are those axioms not agreements? Since you seem to be familiar, what are those axioms?