FPGA-to-HPS SDRAM vs. HPS-to-FPGA

Ok, thanks for that advice. I've dropped the Linux memory to 768M, and so leave the top 256M for my FPGA to access. I then construct an access to that block from within Linux using mmap:

ptr = mmap(NULL, 0x10000000, PROT_READ | PROT_WRITE, MAP_SHARED, fd, 0x30000000);

This works, except that it would seem that access to this memory is MUCH slower than accessing the same chip within Linux address space. So it's not a physical connection problem, but some way Linux deals with memory access via mmap. But mmap should give a virtual pointer to the physical address, and anything else is accessed via a virtual pointer anyway, so why should there be any difference at all?