PCIe root port unable to read endpoint RAM
I've got two boards, both using Cyclone IV FPGAs with hard PCIe IP blocks. One board is a PCIe endpoint and is working great. I can use it in a PC and it's been working well for years.
The other board functions as a PCIe root port and is intended to be used to program and test the first board. I'm having a lot of problems configuring that root port though and so far have been unable to successfully read memory from the endpoint.
I have been able to read and write configuration registers on both the root port (using type 0 config TLPs) and on the endpoint (using type 1 TLPs). Any time I issue a memory read TLP to the endpoint I get a response with no data and completion status of 1 (Unsupported request). I've tried both 32-bit and 64-bit reads and both act the same way.
I know I need to configure both the root port and endpoint config registers and I suspect that this is where my problem lies. Here's what I've done, can anyone tell me what I'm missing:
- Set the root port device control register (0x04) to 6 to enable bus mastering and memory access. I set the slave port device control register to 2 to allow memory access. Note that I always read zero back from these registers no matter what I set them to which is odd.
- Set root port register 0x18 to 0x0100. This sets the secondary bus number to 1 which tells the root port to pass messages addressed to that bus on to the endpoint.
- Set the endpoint base address register at address 0x10. I've tried various addresses but none seem to work.
- Set the PCIe device capabilities register at 0x88 for both the root port and endpoint to a value of 0x0010. That disables all error reporting, enables relaxed ordering, sets max payloads and requests to 128 bytes, disables phantom functions, etc. This is based on the Altera documentation and example code.
- I've tried to set the prefetchable memory base and limit registers in the root port at address 0x24 but for some reason this seems to be read only. The register reads 0x00010001 and can't be set. That LSB is read only and according to the PCI-PCI bridge spec it should be zero so I'm not sure why it's set by the IP core.
- I can write to the prefetchable upper base register on the root port (0x28), but the upper limit register (0x2C) also seems to be read only and stuck at zero. Again, not sure why. I left the upper base register at 0 which I think should allow access to the lower portion of RAM.
Can anyone see what I'm missing here?
Thanks,
Steve