Write from FPGA to external processor's memory

Question

Hi,

I am creating a design that implements a counter (Qsys interval timer) in FPGA and to write these counter values to external processor's memory via PCIe. I memory mapped the counter to PCIe BAR0. Next, I want to run this counter and keep writing to few location in processor's external memory. The read and write request to the counter will be initiated by the processor (master). Once the write request is made, the FPGA should be able to write counter values to the processor's known memory location identified by the memory map. How does the FPGA see the processor address map on PCIe so it knows where to write to ?

In order to make the above design work, I also need help with Qsys configuration and connections.

1. Should PCIe be configured as native endpoint or root port ?

2. I have only three connection between PCIe and counter: pcie-clk to counter-clk, pcie-reset to counter-reset and pcie-BAR0 to counter-slave. Do I need to connect any other PCIe ports to the counter (like cra, txs) ?

Thanks in advance.

altera_forum · Answer

--- Quote Start ---

I am creating a design that implements a counter (Qsys interval timer) in FPGA and to write these counter values to external processor's memory via PCIe. I memory mapped the counter to PCIe BAR0. Next, I want to run this counter and keep writing to few location in processor's external memory.

--- Quote End ---

You are mixing terminology here - which leads me to believe you may not understand PCIe.

BAR0 is what the host processor uses to talk to a PCIe end-point, i.e., your Qsys design.

If your Qsys design is going to access the host (external processor) memory, then it would use the TXS interface. Before the Qsys system is allowed to use PCIe to write to the host memory, you would need to reserve host memory and then get its physical address, which would then be programmed into the TXS address translation register.

--- Quote Start ---

How does the FPGA see the processor address map on PCIe so it knows where to write to ?

--- Quote End ---

The TXS port opens up a 1MB window (or whatever size you configured it to be) on the PCIe address map. Before you write to anywhere in that map, you need to have a driver on the host that allocates memory and then locks it, so that PCIe can access it.

--- Quote Start ---

In order to make the above design work, I also need help with Qsys configuration and connections.

1. Should PCIe be configured as native endpoint or root port ?

--- Quote End ---

If your hardware is plugged into a PCIe slot, then it needs to be an end-point.

--- Quote Start ---

2. I have only three connection between PCIe and counter: pcie-clk to counter-clk, pcie-reset to counter-reset and pcie-BAR0 to counter-slave. Do I need to connect any other PCIe ports to the counter (like cra, txs) ?

--- Quote End ---

You need an Avalon-MM master connected to TXS to perform the 'write' to PCIe. If you want your counter value written to PCIe, then you need to write an Avalon-MM master that generates a PCIe write each time the counter is updated. Alternatively you could just have a NIOS II processor read the counter and then write to PCIe.

Cheers,

Dave

altera_forum · Answer

Hi Dave,

Thank you very much for your feedback.

Is host-processor considered to be Avalon-MM master in this case ?

The PCIe should see the address space on the host processor in order to write to that reserved memory space. Can you tell me if PCIe to host processor address mapping is as per "PCI Express-to-Avalon-MM Address Translation" as mentioned in the Altera's PCIe guide ?

Thanks,

Manish

altera_forum · Answer

--- Quote Start ---

Is host-processor considered to be Avalon-MM master in this case ?

--- Quote End ---

Draw yourself a picture or block diagram and things may become clearer.

* A NIOS processor inside the FPGA can be an Avalon-MM master

A PCIe bridge can contain both PCIe master (initiator) and slave (target), and internal to the FPGA can be both an Avalon-MM master or a slave.

If the NIOS processor or a DMA processor inside the FPGA was to write to PCIe memory on the host then you would have:

* NIOS/DMA Avalon-MM master -> PCIe bridge Avalon-MM slave interface -> PCIe master/initiator performs the PCIe transaction.

If the host processor was performing an access to an Avalon-MM slave, eg., a register inside the FPGA, then you would have:

* Host processor PCIe master -> PCIe bridge PCIe slave interface -> Avalon-MM master interface (part of the bridge) -> accesses the Avalon-MM slave register

Each time you go through a bridge from one protocol to another, you can modify the address.

--- Quote Start ---

The PCIe should see the address space on the host processor in order to write to that reserved memory space. Can you tell me if PCIe to host processor address mapping is as per "PCI Express-to-Avalon-MM Address Translation" as mentioned in the Altera's PCIe guide ?

--- Quote End ---

"The PCIe" means what? Again, ask clear questions, and it'll make providing clear responses easier.

Here's what I think you are asking:

For the Altera PCIe-to-Avalon-MM bridge, what is the PCIe Initiator Address mapping?

Or another way, if I perform an Avalon-MM access to the TXS slave of the bridge, how does that translate to a PCIe address?

The answer is; the low address bits are taken from the Avalon-MM address, and the MSBs are set by a register in the bridge. The MSBs need to be set by your software, after the host allocates and locks some memory into the PCIe address map (as viewed by all the devices attached to the same PCIe bus your board is plugged into).

So what is going on, why is this needed? Consider the following; your board is plugged into a PCIe bus with a 64-bit address, and you are a 32-bit processor ... how can you "see" the 64-bit PCIe address map? The answer is, that you cannot, you can only see a small window into the PCIe address map. If your window was 32-bits, then you would be able to "see" 4GB of the PCIe address map, but you would have no space for any other Avalon-MM devices. The Qsys PCIe bridge allows you to create several small windows, eg., some windows with a default size of 1MB.

Take a look at this Qsys/PCIe thread. There's a document there that might help:

http://www.alteraforum.com/forum/showthread.php?t=35678

Cheers,

Dave

altera_forum · Answer