seperate memories for data and instruction masters?

The instruction master will only read instructions! For a normal compilation this is all that is in .text.

Everything else is read by the data master - including the .rodata that contains the jump tables for some switch statements.

So you may want to arrange that the linker puts .rodata in the data memory, not with the .text.

Then you technically don't need data access from the nios cou to the instruction memory. But ...

The linker may also be placing the initialised data immediately after the .text and adding code to copy it to the relevant areas. This will require read (data) access to the code memory. Or use a different linker script etc.

You also need to be able to get your code into the instruction memory!

The JTAG loader writes everything from the Nios cpu - so needs write access to the code memory.

If you are loading from some external interface (eg PCIe) then the external interface needs MM-Avalon bus access to both the code and data memory block. The nios cpu doesn't need MM access to the data area - so it can have the same address for both masters.