Simple PCIe DMA

The only two 64-bit DMA engines that I know of are these:

http://www.altera.com/support/refdesigns/ip/interface/ref-pciexpress-avalonmm-hp.html

http://www.alterawiki.com/wiki/modular_sgdma

That first one comes with a Linux driver and the second one has a Linux driver up on the wiki here: http://www.alterawiki.com/wiki/linux_pcie_driver

I looked around for some third-party IP at one point and came across a PCIe IP core called "Lancero"

http://www.lancero.eu.com/lancero-pcie-sgdma/

http://www.logicandmore.com/lancero.html

http://www.microtronix.com/ip-cores/lancero-sgdma

I *have not* looked at this core or evaluated it ... but it looked interesting ...

Cheers,

Dave

--- Quote Start ---

I think the one we are currently using maps a chunk of Avalon address space to PCIe physical addresses by internally generating the high address bits - I don't see how this is of any use at all!

--- Quote End ---

Why not?

If you're using the memory mapped interface then DMA target addresses use the high bits of the local address to determine which

page of the address translation table to lookup to get a host address. It's simple enough and works well, all you have to do is set up

the address translation table the way you want it.

If you need multiple sources you'll just have to multiplex them somehow so that transfers are done in whatever sequence you need.

The logic to do this can be pretty trivial.

Nial.

Hi Nial,

--- Quote Start ---

It's simple enough and works well, all you have to do is set up the address translation table the way you want it.

--- Quote End ---

The original poster wanted to be able to DMA to an arbitrary 64-bit address, and he wanted 10 different DMA engines. As far as I am aware, the TXS port on the Qsys PCI IP core would only allow one value for the 32 MSBs of the address, so the 10 DMA controllers would be restricted to a 4GB region of the PCIe memory map. Its pretty common now for host PCs to have in excess of 4GB, so a driver on the host could easily provide PCIe DMA target page addresses that cross a 32-bit address boundary.

Bottom line is that the Qsys PCIe core PCIe bus master interface is too simplistic.

Cheers,

Dave

I could manage with physical addresses below 4G.

But my attempts to configure the PCIe avalon MM master that way fail - unless I set the master interface to 64bit - and the dma controller doesn't want to generate 64 bit slave addresses.

I can't actually see how the dma controller can actually generate long TLP for reads at all.

--- Quote Start ---

I could manage with physical addresses below 4G.

--- Quote End ---

Ok (your device driver on the host end would have to deal with it).

--- Quote Start ---

But my attempts to configure the PCIe avalon MM master that way fail - unless I set the master interface to 64bit - and the dma controller doesn't want to generate 64 bit slave addresses.

I can't actually see how the dma controller can actually generate long TLP for reads at all.

--- Quote End ---

The Qsys PCIe core (which it sounded like you were talking about) does not have a DMA controller. If using the Qsys PCIe core, you would then use a regular Avalon-MM DMA controller on the Avalon bus, which would use 32-bit DMAs to the TXS port of the Qsys PCIe core.

This thread has links to documents and code on how that core can be configured:

http://www.alteraforum.com/forum/showthread.php?t=35678

Cheers,

Dave

To answer myself 4 years later....

We set the Qsys avalon to PCIe bridge for 512 16k pages and allocate non-contiguous kernel buffers that don't cross 16k boundaries.

These can me mmap()ed into contiguous user addresses ('interesting' on windows!).

All the used entries point to a 'dummy' page 'just in case'.

Any Avalon master has to do burst transfers to generate reasonable sized TLP.

I ended up writing a suitable DMA controller.