@johnjunk: Thanks for the heads-up, I'll check it out.
I think I've found the problem, or a major contributing factor... we support multiple output modes and therefore we must dynamically configure the PLL used for video output. Previously I was configuring the PLL *after* the VIP components were brought out of reset, and then starting the CVO module via the control port.
If I configure the PLL and *then* reset the entire VIP core (including CVO), and subsequently start CVO, the problem is all-but-eliminated.
Still more investigation required...
EDIT: is anyone else using dynamically-configured PLLs for VIP/CVO?