VIP Scaler II Megacore: Garbage at bottom of image

The bilinear and polyphase filters require access to multiple pixels vertically. You have 8 taps vertically so that should require 4 lines to be buffered in the scaler before output data can begin. I'm guessing that is the cause of your underflow. In your video output there are 42 lines of vertical blanking so if you trigger the process that feeds data into the scaler near the start of the output blanking (vs. at the start of the active portion of the output) then you should have plenty of margin to have the scaler spitting out pixels before the active portion of the display output.

So I think this is probably just an issue of when you kick off the scaler input process relative to the output display timing. I've been using the VIP scaler (I & II) for the last 5+ years and never had any problems with it.