You would constraint yourself to a subset of C++ which is not C. For example, forbid all raw pointers and only use smart pointers. Now your memory is freed automatically via RAII and use-after-free errors are much harder to create.
As you said, this is a subset of C++. But everyone debates what the right subset should be. In reality C++ is incapable of solving the problem, it can only propose good practices.
Though if one only uses smart pointers they will have to accept the performance hit that that brings -- in high performance situations, this may be unacceptable.
Circumstances where using smart pointers affects performance are vanishingly rare. Where performance matters, on any critical path no allocation or deallocation is done, so a smart pointer can have no effect. On other paths, any difference is swamped by overhead in malloc and free.
There's some argument to be made about bookkeeping and (maybe? I'm not sure) an extra level of indirection with shared_ptr, but the cost of unique_ptr is literally zero.
Passing unique_ptr by move works out to be surprisingly expensive, so one doesn't, where that matters. (It is usually a mistake to have unique_ptr visible in an interface at all, but passing a pimpl type, implemented with unique_ptr, by move costs exactly the same.)
And of course no one who cares for their reputation is seen using shared_ptr.
My guess is that because in most common x86_64 calling conventions the caller is responsible for destructing the parameters, it has to run the unique_ptr parameter destructor which needs to check if it was moved from in the callee or not. Additionally, because this destructor is not a trivial one (i.e. it does something), unique_ptr cannot be passed directly through a register but must be spilled on the stack.
This is correct. Chandler Carruth gave a talk about unfortunate pessimizations imposed by the ABI. Where the call is inlined, that stuff is optimized out, but functions that take a moved unique_ptr or pimpl object -- taking ownership of a heap object -- tend not to be inlined unless they mostly just pass it along to one that isn't inlined.
So, the cost of moving a smart pointer onto the stack and, presumably, off again to someplace less ephemeral might matter, on a critical path. But it would be a mistake to exaggerate this: we are talking about a couple of unfortunate ordinary memory operations, not traps, function calls, or atomic synchronizations, when you might have hoped to be using just registers. If you are already touching memory -- something pretty common -- it is just more of that.