youtube-dl -q -o - 'url' | castnow /dev/stdin
I'm guessing it is trying to seek, or running a stat first to check filesize or similar.