HD SDI and Clocked Video Input issues

Sounds like you're making some progress. 150 MHz might be cutting it a bit too close, considering the overhead of the Avalon bus. I'd try upping it, if possible.

Also, how big is the FIFO in the CVI, it might not be big enough to cope with the backpressure from the Scaler. You might also want to experiment with putting framebuffers before and after the scaler (only one of them needs to be triple buffered) if you can afford the bandwidth. I've had success with that in the past. If you're only ever going to scale down to 640x480, you may be able to get away with just a large FIFO in the CVI and a framebuffer downstream of the scaler.