Not the death of optimism, just the winning of the NASA bureaucracy in keeping the organization size by pushing the "cheaper" space shuttle.
With the benefit of hindsight, flying into space with the X-15's successors would have happened if there was not a need to beat the Soviets quickly with rockets.
Also throwing away the Saturn rockets after around 20 launches was just silly.