I'd add a very important thing - with your primal problem you are trying to e.g....

I'd add a very important thing - with your primal problem you are trying to e.g. minimize a function and you proceed the way that all your intermediate solutions are feasible. With dual approach you flip the direction, i.e. maximization in this case, but you start in completely infeasible solutions and hope to end up in the very first feasible solution that should be your optimum.

Now how does that relate to automatic differentiation I am not sure either.