PCIe Hard IP returning stale data for Cyclone IV

I wrote a simple test from RC ( Linux ) -> Endpoint Arria V with Hard PCIe.

The objective of the test is to write to IMEM which is via BAR1 and immediately read back the same location and compare, then move

on to the next IMEM location.

( My simpler test will just park on a single IMEM address and itterate through different 32 bit patterns.

Observation: The test passes fine prior to running the NIOS II ie just after the Linux DD is installed .

After the NIOS II code is running , runs out of a different IMEM, the above test fails where the data read corresponds to the data written two cycles back.

I believe NIOS is running out of its own IMEM for code, stack , heap. So .. is ther some interaction on the Avalon MM interconnect. The NIOS II is polling

memory in the RC (Linux memory ).

Do I need to do something for the PCIe BAR masters to make them strictly ordered ... of something to strictly order Avalon MM interconnect ?

Thanks, Bob.