It's actually more likely that Jesus as an out of wedlock child that pre-dates Joseph. Jesus was referred to as "Jesus son of Mary" not Joseph, which was the custom at the time for illegitimate children. It particularly contrasts with how his brothers were referred to (yes, he had brothers, and probably sisters).
Speaking of which, it's striking that James "the Just", brother-of-Jesus, first-patriarch-of-the-Jerusalem-Church never mentioned the virgin birth. Ever.
> It's actually more likely that Jesus as an out of wedlock child that pre-dates Joseph.
I'm not sure they would have drawn a line between the two at that point (hence why I as too lazy to look up when they wed). Shotgun weddings were not considered having children out of wedlock for western europe, and further back you see it's more of an economic agreement to father all the woman's children. In fact, it's entirely possible marriages would have been more of de-facto arrangements based around paternity for non-arranged marriages (which are economic transactions).
EDIT: it's insinuated he's upset because she's pregnant and they haven't had sex, but they had already been married. Again, at this point, it's more of a question of semantics of the time as to whether it counts as adultery or pre-marriage pregnancy. Historically, there may be other evidence to point one way or the other, but I don't think it matters much to joseph (or to christians).
18 Now the birth of Jesus Christ was on this wise: When as his mother Mary was espoused to Joseph, before they came together, she was found with child of the Holy Ghost.
19 Then Joseph her husband, being a just man, and not willing to make her a public example, was minded to put her away privily.
It is even more likely that he was just an ordinary legitimate child of Mary and Joseph, and the whole virgin-birth story developed later (after the death of Jesus) among Greek-speaking early Christians, in order to tie in to the Isiah-prophecy.
