VGA controller AppNote 333

Check your vertical and gorizontal timing parameters.

For 800*600 it should be next:

// horizontal timing parameters

parameter hsync_end = 80;

parameter hblank_begin = 200;

parameter hblank_end = 1000;

parameter hline_end = 1040;

// vertical timing parameters

parameter vsync_end = 4;

parameter vblank_begin = 26;

parameter vblank_end = 626;

parameter vframe_end = 629;

And when you send data by dma, dmalength must be 800*600*4.