Say no to OpenCV, unless it got mayor redesign. Its great for experimenting and prototyping, just like GNU Radio Companion, but its nickname is 'lets copy that buffer one more time'.
First thing you do when serious about image processing is you DO NOT deploy OpenCV on the end product. You settle on algorithm that works for you, and reimplement it in C (or verilog) as close to metal as you can get.
