Try signaltapping the write and read master interfaces and trigger on the read and write signals. It's possible that they are issuing an incorrect access to the memory causing problems. The memory controller will only work if the master issues read or write transactions that are within the Avalon-MM spec (i.e. could be a master bug).
Once you get it captured attach the file to this post or take a screenshot with the first read/write captured and zoomed in on. I should be able to look at it and tell you if the master is staying within spec on the first access.