Best way to interface my CPU

I have another question, this one is a bit off topic:

I am implementing the framebuffer to display to a 640*480 vga connection at 25mhz with 4 bits for each colour. I am wondering how I am going to implement DMA to it. I can't use the onchip FPGA memory bits to store it all because I won't have enough bits to store it all. How can I design it to work in this scenario?

EDIT: I quickly did some calculations and it looks like I can implement the DMA to my DE1's SRAM, and it should be fine. Now I just need to figure out how to interface to the SRAM. Also should I read directly from SRAM to the VGA controller or should I implement a FIFO queue for extra safety.