First, direnv (along with nix-direnv) is really the glue that makes all of this work seamlessly.
It depends on what the dotfiles are for and how they're used.
Here's an example of a problematic one: AWS configs. You can theoretically override the default location of these files with environment variables, but a lot of tooling doesn't respect this and will break as a result.
I often deal with a large number of AWS accounts, so my solution is to have one main way for populating AWS config files that lives outside of projects, and then, because I have a rubric for account/role assignment naming, I can select the correct account/role for each project, and I can even have projects that switch them based on specific code/deployments.
Can you say what specifically? Many language-specific package managers match your description and are completely handled by Nix, but maybe you're talking about something else?
I am aware of home-manager but am not sure how (or if) it would work for per-project dot file management.