SOPC builder - connect a custom component to tightly coupled i/f?

Depending on the amount/type of debugging you need to do I think there is a JTAG -> avalon master block somewhere which might be enough for you.

I can imagine a system where you expose the nios soft reset bits, and have a piece of vhdl that copies code/data from a pre-initialised memory block into the nios code/data memory areas prior to removing the soft reset.

You might then be able to use the jtag->avalon master block to update that 'pre-initialised' memory area and force a reset for development build and load.