That historical detail is actually somewhat irrelevant to the quote. This is from "Either/Or" and Kierkegaard was making a point about the nature of decision-making and regret, not about marriage. The fuller quote:
"If you marry, you will regret it; if you do not marry, you will also regret it; if you marry or if you do not marry, you will regret both. Laugh at the world’s foolishness, you will regret it; weep over it, you will also regret it; whether you laugh at the world’s foolishness or weep over it, you will regret both. Believe a girl, you will regret it; if you do not believe her, you will also regret it; if you believe a girl or you do not believe her, you will regret both. [...] This, gentlemen, is the sum of all practical wisdom."
And later:
"Were I to wish for anything I would not wish for wealth and power, but for the passion of the possible, that eye which everywhere, ever young, ever burning, sees possibility. Pleasure disappoints, not possibility."
"If you marry, you will regret it; if you do not marry, you will also regret it; if you marry or if you do not marry, you will regret both. Laugh at the world’s foolishness, you will regret it; weep over it, you will also regret it; whether you laugh at the world’s foolishness or weep over it, you will regret both. Believe a girl, you will regret it; if you do not believe her, you will also regret it; if you believe a girl or you do not believe her, you will regret both. [...] This, gentlemen, is the sum of all practical wisdom."
And later:
"Were I to wish for anything I would not wish for wealth and power, but for the passion of the possible, that eye which everywhere, ever young, ever burning, sees possibility. Pleasure disappoints, not possibility."