Strange problem: design works with SignalTap, but not without it.

I'm running into a strange problem that I hope someone out there can help me out with. I had a system that consisted of a single SOPC System that instantiated two NIOS II/f CPUs. The CPUs do not share any address space -- the primary one has a dedicated SRAM interface and the other is small enough to have code reside entirely in on-chip memory. The two CPUs run autonomously with the exception of two FIFOed UARTs that I implemented so they could communicate to one another for testing purposes. All I've done is to swap the tx/rx lines so they can communicate. This system worked just fine.

The plan is to eventually drop one of the CPUs when we get closer to production. So, I took the secondary one out of the "main" system and created a new SOPC System. The gives me two SOPC systems: a 'primary' one and a 'secondary' one. These two are instantiated by a top-level VHDL file that brings them together and connects them to off-chip peripherals.

Here's where the strangeness sets in. In this two SOPC System configuration, I cannot use nios2-download to successfully download the primary CPU's .elf file. The secondary CPU's .elf file loads correctly. As mentioned above, the primary has code allocated in SRAM. When I try to download, there is a consistent verification error in a section of the SRAM. In other words, because the .elf is not successfully verified, the processor is left in a paused state.

To try and trouble-shoot the problem, I added in Signal Tap II. I configured a single signal to trace the reset line to begin with (assuming there may be a reset problem or something) and gave it 32k samples. I built this, deployed it to the target and hit run. I then loaded in my primary CPU's .elf file. No problems. It all verified consistently and successfully ran.

So, I removed Signal Tap. Rebuilt. Redeployed. Tried to load the primary CPU's .elf file -- verification failure. Huh? Added Signal Tap back in -- I can now load the .elf file successfully.

What gives? Is it a floor planning issue? Is it a timing issue? We had an Altera FAE on-site that helped us build our SDC file. I can only assume that's correct. I do have to admit that the SDC file was developed prior to our adding the secondary CPU so it still could be potentially incorrect for this configuration.

Any and all help would be appreciated.

Thanks,

--tim