Because of quantum mechanics this is a genuinely open philosophical question. The equations which time evolve the wave function are time-symmetric, which means that even if you know the current state of the universe's wave function somehow, you still only get a probabilistic projection for the state in the past. I think this leaves open the genuine physical possibility that the past does not exist in the same way that the present exists. A lot comes down to what you think about the ontology of quantum mechanics.
This has nothing to do with fiction. Roughly speaking, if you are some kind of wave function realist, then the implication that the past has a similar kind of uncertainty as the future is a simple consequence of the time reversibility of quantum mechanics (without the measurement postulate). Read a book.
