For the Echo, at least, it has to use your home network, so you could pretty easily run a packet capture to see if it's ever sending audio out when you don't want it to.
It's a cat-and-mouse game: What if it only sent the clandestine information when it picks up the "normal" word? The point is you don't control the device or its software.