I have released a Linux device driver for Altera PCI Express Chaining DMA reference design. I am currently trying to get this in newer kernel releases. The patch can be found here: http://marc.info/?l=linux-kernel&m=122813921631142&w=2 It has been tested on x86 PC and PowerPC platforms. Regards, Leon Woestenberg.

I recently started getting involved using an Altera Aria IIGX for an embedded design. I am using the PCIe with chaining DMA on the devel board to test out my Linux driver. I use the same techniques as the Linux driver posted here, but with a difference. We use high end server class PC's with lots of memory (8G or more), so we use the PAE kernel. This means that it is very possible that buffers needed for DMA exist above 4GB, so different calls must be used to assemble the SG list, specifically pci_dac_page_to_dma(), which provides the pages as a 64 bit entity (dma64_addr_t). All this works great except when programming a descriptor with any non-zero value in the upper address field of the descriptor table. The resulting DMA from the DMA engine should issue a 4DW read request providing a 64 bit address, however it does not, it still issues the 32 bit address 3DW read request instead. I have PCIe protocol analyzer traces to prove it. I have submitted this to Altera, but have not received an answer yet. This same type of problem would exist with a 64 bit kernel.

I had a similar issue and found that the reference design DMA hardware has a parameter set that causes it to assume 32-bit addressing, and thus, always generates 3DW headers. I worked around the issue by changing the parameter RC_64BITS_ADDR in altpcierd_example_app_chaining.v from 0 to 1. After rebuilding the h/w in Quartus, I observed 4DW headers where required. I hope this helps.

BrettFavre, Thanks! This indeed did the trick. I now see it properly addressing up past the 4G barrier.

OK, so I now have it properly addressing with 64 bit addresses, but there is a problem with actually verifying data transfer. I can DMA a block of data to it, for example the following is part of a PCIe analyzer trace for a 256byte transfer. The first two TLP's are reads from the RC and show the correct data which is a simple incrementing pattern.. TLP 1 05550000 05550001 05550002 05550003 05550004 05550005 05550006 05550007 05550008 05550009 0555000A 0555000B 0555000C 0555000D TLP 2 0555000E 0555000F 05550010 05550011 05550012 05550013 05550014 05550015 05550016 05550017 05550018 05550019 0555001A 0555001B 0555001C 0555001D 0555001E 0555001F But when I read it back using DMA, the data is corrupted.. 05550000 05550001 05550002 05550003 05550004 05550005 05550006 05550007 05550008 05550009 0555000E 0555000F 05550010 05550011 05550012 05550013 05550014 05550015 05550016 05550017 05550018 05550019 0555001A 0555001B 0555001C 0555001D 0555001E 0555001F 00000000 00000000 00000000 00000000 The ones highlighted in RED show where the errors start. Basically the 11th value which should be 0555000A is actually 0555000E. 0555000A - 0555000D are missing. It almost looks like the data starting at 0555000E overwrote it at the wrong spot as if it had a bug incrementing the internal memory address.

from where i can download the pcie driver for linux? can you provide any link for that?

Linux device driver for Altera PCI Express Chaining DMA reference design

11 Replies

Altera_Forum
Honored Contributor
16 years ago
I recently started getting involved using an Altera Aria IIGX for an embedded design. I am using the PCIe with chaining DMA on the devel board to test out my Linux driver. I use the same techniques as the Linux driver posted here, but with a difference. We use high end server class PC's with lots of memory (8G or more), so we use the PAE kernel. This means that it is very possible that buffers needed for DMA exist above 4GB, so different calls must be used to assemble the SG list, specifically pci_dac_page_to_dma(), which provides the pages as a 64 bit entity (dma64_addr_t). All this works great except when programming a descriptor with any non-zero value in the upper address field of the descriptor table. The resulting DMA from the DMA engine should issue a 4DW read request providing a 64 bit address, however it does not, it still issues the 32 bit address 3DW read request instead. I have PCIe protocol analyzer traces to prove it. I have submitted this to Altera, but have not received an answer yet. This same type of problem would exist with a 64 bit kernel.
Altera_Forum
Honored Contributor
16 years ago
I had a similar issue and found that the reference design DMA hardware has a parameter set that causes it to assume 32-bit addressing, and thus, always generates 3DW headers. I worked around the issue by changing the parameter RC_64BITS_ADDR in altpcierd_example_app_chaining.v from 0 to 1. After rebuilding the h/w in Quartus, I observed 4DW headers where required.

I hope this helps.
Altera_Forum
Honored Contributor
16 years ago
BrettFavre,

Thanks! This indeed did the trick. I now see it properly addressing up past the 4G barrier.
Altera_Forum
Honored Contributor
16 years ago
OK, so I now have it properly addressing with 64 bit addresses, but there is a problem with actually verifying data transfer. I can DMA a block of data to it, for example the following is part of a PCIe analyzer trace for a 256byte transfer. The first two TLP's are reads from the RC and show the correct data which is a simple incrementing pattern..

TLP 1
05550000 05550001 05550002 05550003 05550004 05550005 05550006 05550007 05550008 05550009 0555000A 0555000B 0555000C 0555000D
TLP 2
0555000E 0555000F 05550010 05550011 05550012 05550013 05550014 05550015 05550016 05550017 05550018 05550019 0555001A 0555001B 0555001C 0555001D 0555001E 0555001F

But when I read it back using DMA, the data is corrupted..

05550000 05550001 05550002 05550003 05550004 05550005 05550006 05550007 05550008 05550009 0555000E 0555000F 05550010 05550011 05550012 05550013 05550014 05550015 05550016 05550017 05550018 05550019 0555001A 0555001B 0555001C 0555001D 0555001E 0555001F 00000000 00000000 00000000 00000000

The ones highlighted in RED show where the errors start. Basically the 11th value which should be 0555000A is actually 0555000E. 0555000A - 0555000D are missing. It almost looks like the data starting at 0555000E overwrote it at the wrong spot as if it had a bug incrementing the internal memory address.
Altera_Forum
Honored Contributor
15 years ago
from where i can download the pcie driver for linux?
can you provide any link for that?
Altera_Forum
Honored Contributor
15 years ago
soni1877

See the first post in this thread for a publicly released driver.

My driver is proprietary company property.
Altera_Forum
Honored Contributor
15 years ago
rpoffen,
the link in first post is a patch not the base code.
i downloaded one code from
git (as its my second post i cant paste links) but that doesn't seems to be correct as per the patch you are telling
is it the exact driver code for pci express?
Altera_Forum
Honored Contributor
15 years ago
Hi soni1877,

It is the complete driver code, it appears to be provided as a patch to a Linux distribution. If you want to make standalone files, it should not be hard to extract the code into a plain .c file. If nothing else, just copy the text of the patch into the respective file (altpciechdma.c) and edit it to remove the '+' characters at the front of the lines and the stuff before the actual C code.
Altera_Forum
Honored Contributor
14 years ago
--- Quote Start ---
OK, so I now have it properly addressing with 64 bit addresses, but there is a problem with actually verifying data transfer. I can DMA a block of data to it, for example the following is part of a PCIe analyzer trace for a 256byte transfer. The first two TLP's are reads from the RC and show the correct data which is a simple incrementing pattern..

TLP 1
05550000 05550001 05550002 05550003 05550004 05550005 05550006 05550007 05550008 05550009 0555000A 0555000B 0555000C 0555000D
TLP 2
0555000E 0555000F 05550010 05550011 05550012 05550013 05550014 05550015 05550016 05550017 05550018 05550019 0555001A 0555001B 0555001C 0555001D 0555001E 0555001F

But when I read it back using DMA, the data is corrupted..

05550000 05550001 05550002 05550003 05550004 05550005 05550006 05550007 05550008 05550009 0555000E 0555000F 05550010 05550011 05550012 05550013 05550014 05550015 05550016 05550017 05550018 05550019 0555001A 0555001B 0555001C 0555001D 0555001E 0555001F 00000000 00000000 00000000 00000000

The ones highlighted in RED show where the errors start. Basically the 11th value which should be 0555000A is actually 0555000E. 0555000A - 0555000D are missing. It almost looks like the data starting at 0555000E overwrote it at the wrong spot as if it had a bug incrementing the internal memory address.

--- Quote End ---

I am having a similar problem with the 64 bit PCIe ref design on a Stratix IV GT FPGA. I wonder if you have had a solution for it. It seems to me that the problem is related to the Avalon-ST interface between the PCIe hard IP core and the application on endpoint.
Altera_Forum
Honored Contributor
13 years ago
Hi hw342,

I have a host PC running linux (Ubuntu 10.04 LTS 64) and connected to an DE4 Stratix IV GX over PCIe. I explore the use of the FPGA as co-processor, but i get same problem of yours.
Do you find same solution?

Best Regards

Forum Discussion

Linux device driver for Altera PCI Express Chaining DMA reference design

11 Replies

Recent Discussions

Agilex 7 slew rate reconfiguration

Agilex-7 AXI MCDMA for PCIe hang

Constraints not being picked for DCFIFO

Can't generate F-Tile Ethernet Hard IP Design Example

MAX10 TSE reference design