I can give one thing for you to ponder: what's the true difference of the world you experience in the waking state and the world you experience during dreaming state and the cause behind both of these? The puzzle of the universe, infinity and multiverse can become apparent if you ponder over the above question.
The world we experience in the waking state is persistent and objective. The world we experience during the dreaming state is subjective and temporary. There, I pondered, and it did not make “the puzzle of the universe, infinity and multiverse apparent”.
The objectivity you are assigning to the world of waking state is also subject to your mind. Different kinds of minds experience the world differently. Scientific investigation confirms us that the object we experience do not exist like that; it's just set of atoms which are empty by their nature. There's nothing solid or liquid in actual, it's our mind way of interpreting the input from senses that makes us experience solidity/etc.
Regarding the persistency argument, the waking world seems persistent to you because it has a longer duration relative to your dream. Sometimes we dream in a dream and experience continuity of the dream-world as well. There are stories of people experiencing years of life in dreaming [0] which confirms that continuity in dream is also possible.
That said, I'm not trying to prove that waking and dreaming state are literally the same. The idea is to ponder on the generality behind waking and dreaming states and question/contemplate on the true nature of reality. Do we not observe that it's the mind which seems to be the cause of us experiencing both these states the way we experience them? If so and if the dream seems to be our personal creation then the question arises what if the waking world is a our collective (cosmic) mind dream? If it's a cosmic dream then we can start deducing answers such as there cannot be any boundary to the universe because it's all in mind as a reflection. And multiverse is also possible as the same cosmic mind can dream N number of worls/universes as it's not bound by any real material quantity.
There have been interesting books [1] written on this subject going very deep into above idea which also talk about methods to realize it as experience rather than just as an intelectual exercise.
Our awake experience is also subjective. When we look at a car, we see how it slides on 4 wheels. A deeper look reveals that the wheels aren't sliding, but rolling. What we see in our mind is simplified models, while the objective reality is much more complex, messy and not comprehensible.
