Honestly, for complex ui like this, I prefer to write it as a separate frontend app written in react/typescript. Django template system is great for normal websites (e.g. news/blog/ecommerce websites), but for complex application ui you'll start encountering pains like you mentioned. Implementing complex ui with a lot of interactivity is just easier in react. The drawback is now you have to implement an api layer to bridge django and react, which actually not that bad using django rest framework or even vanilla django views.
