ddr2 high-performance controller to slow?

It sounds the SDRAM is being exposed as a x64 slave port with burst length of 2 to the fabric. If your master is non-bursting then that means you will at most be able to achieve 50% efficiency of the memory total bandwidth. I'm not sure how efficient the memory controller will be if you set the local burst count to 1 so maybe these steps would work best for you:

1) Configure the memory controller for half rate operation (this will double the width of the local side to 128-bits, burst length of 1, and half the clock rate)

2) Configure your master to operate on (non-bursting) 128-bit data

3) If there are multiple masters competing for the memory, increase their arbitration shares so that multiple sequential accesses can get through