The manager isn't owed the option of declining OP's plans. The manager can decline to give leave, but if it's simply to prevent OP from attending something that the manager disapproves of, then it's sheer inappropriate cattiness.
Unless OP is at risk of exposing privileged information about their company, the manager has no business trying to interfere with OP.
OP can go to conference, in his/her "own time". OP is an adult and does not need an OK from manager to go to the conference in his/her private life.
But, OP cannot expect the manager to give the OK. OP is not "entitled" to an automatic OK. The manager does not owe OP an OK.