If you give the TV internet access it will still identify the content and share it with advertisers, even if you are using an external box. If you don’t give it access, some have been observed connecting to open networks nearby.
Beware of any nearby open WiFi network, as the TV will try to use it, and don't forget HDMI cables with Ethernet compatibility; if the TV doesn't connect to the Internet, the external box almost certainly will, and the TV would immediately use it to get online as well.
What good does unplugging it when it’s not in use provide? Surely it wants to share information about how you use it not how you weren’t using it. I guess if it has a built in vouce assistant or something but that’s not something hard to avoid.