I'm wondering whether you could apply an edge-detect filter to your input frames and combine it with the hue-based detection. I'd assume that irrespective of the ball color, an edge detect + posterize would give you a video frame with clear spots where the balls are, which should be relatively easy to find. I remember from the image analysis courses I took in college that spot detection is a very common operation in automatic analysis of medical images, and hence extensively researched. After you know where each ball is you could convert the frame to YUV and use the UV (chroma) channels to detect the ball colors.
Edge detection algorithms are fairly slow and noisy from what I found. To do this in realtime you have to have fast algorithms or cut down on the information to process over.
