Nuclear reactors are expensive mostly because the regulatory requirements make them expensive. There is a lot of room for advanced nuclear (e.g. gen III reactors) that can address a lot of the problems and bring down the costs. Unfortunately, we don't have a clear regulatory pathway here yet, but there is some progress being made recently.
