How to Implement VHDL Drawing Buffer in DE2

You can use pipeline or burst transfers to read 32 bits per clock cycle. Then put a FIFO between your memory reader and the VGA output, so that you can use some cycles to write to the memory without interrupting the output flow.