No. OpenAI would not invest their operating capital in startups - that would be legally and fiscally irresponsible. Instead, a separate entity is created to manage the fund and OpenAI controls that entity. That fund then raises money from LPs specifically for investments. I would imagine some of the profits/carry will then flow back into the parent. What this ends up being though is mostly a branding exercise to boost OpenAI's developer ecosystem, with some upside for the company itself if they are able to fund some successful startups.
Keep in mind that the former president of YCombinator is the CEO of OpenAI. That might help explain a move like this. Also recently a lot of people left Open AI at the same time. There was some speculation that they are going to start a new startup. Maybe this is a way for OpenAI to be able to invest in them.