How to Implement VHDL Drawing Buffer in DE2

I'm not quite sure how the processor will get me past the lack of write cycles in the display operation unless it uses the "porch" time to write to the display.

The trouble I am having is that I need to supply a signal at 25 MHZ (640x480x3x8), but with a system clock at 50 MHZ I can only supply (640x480x1x8 2 clks per memory access time). And doing this will leave me no time to write to memory (except for 10% porch time)!