Hi everyone, i'm using an Arria V starter kit for PCIe application and i used the reference design posted in altera wiki. The application software reported success in one PC but not in other with different features. The driver code is installed in two with success but in one, user application reported timeout for read, write and simultaneous RW. Had anyone has the same problem? Have anyone some suggestion about what can i do to try solve this problem?

Hi there! I am having the same issue with timeout being reported for read and write. Have you found a solution for it? This is the output I see. I am using the driver for linux. ********************************************** ** ALTERA AVMM DMA driver ** ** version 2.02 ** ** 1) start DMA ** ** 2) enable/disable read dma ** ** 3) enable/disable write dma ** ** 4) enable/disable simul dma ** ** 5) set num dwords (256 - 2048) ** ** 6) set num descriptors (1 - 128) ** ** 8) loop dma ** ** 10) exit ** ********************************************** Run Read ? 1 Run Write ? 1 Run Simultaneous ? 0 Read Passed ? 0 Write Passed ? 0 Simultaneous Passed ? 0 Read EPLast timeout ? 1 Write EPLast timeout ? 1 Number of Dwords/Desc : 512 Number of Descriptors : 128 Length of transfer : 256 KB Rootport address offset : 0 Read Time : 1 s and 366248 us Read Throughput : 0.000179 GB/S Write Time : 1 s and 361611 us Write Throughput : 0.000179 GB/S Simultaneous Time : 1 s and 362982 us Simultaneous Throughput : 0.000358 GB/S

I am encountering the same problem, in which my DMA read and writes always time out. ************************************************ ALTERA AVMM DMA driver ** ** version 2.02 ** ** 1) start DMA ** ** 2) enable/disable read dma ** ** 3) enable/disable write dma ** ** 4) enable/disable simul dma ** ** 5) set num dwords (256 - 2048) ** ** 6) set num descriptors (1 - 128) ** ** 8) loop dma ** ** 10) exit ** ********************************************** Run Read ? 1 Run Write ? 1 Run Simultaneous ? 1 Read Passed ? 0 Write Passed ? 0 Simultaneous Passed ? 0 Read EPLast timeout ? 1 Write EPLast timeout ? 1 Number of Dwords/Desc : 512 Number of Descriptors : 128 Length of transfer : 256 KB Rootport address offset : 0 Read Time : 1 s and 227400 us Read Throughput : 0.000199 GB/S Write Time : 1 s and 230367 us Write Throughput : 0.000198 GB/S Simultaneous Time : 1 s and 238158 us Simultaneous Throughput : 0.000394 GB/S When I tried the burst mode (loop dma), I got # DMA data error! Type in dmesg to show more details! And by checking dmesg I got the following Read DMA times out DWORD = 00000100 Desc = 00000080 Read DMA times out DWORD = 00000100 Desc = 00000080 Read DMA times out DWORD = 00000100 Desc = 00000080 Read DMA times out DWORD = 00000100 Desc = 00000080 Read DMA times out DWORD = 00000200 Desc = 00000080 Write DMA times out DWORD = 00000200 Desc = 00000080 Simultaneous DMA times out DWORD = 00000200 Desc = 00000080 command issued from user space doesn't exist 2608 command issued from user space doesn't exist 2608 Does anyone have any clue why this would happen? Any suggestion or hint is much appreciated!

Have you checked if the FPGA has been programmed successfully and if the board has been enumerated properly in the PC? If using Linux, run the lsmod command and check if the driver is loaded. Have you checked if the device ID in the driver matches with the one in the board?

--- Quote Start --- Have you checked if the FPGA has been programmed successfully and if the board has been enumerated properly in the PC? If using Linux, run the lsmod command and check if the driver is loaded. Have you checked if the device ID in the driver matches with the one in the board? --- Quote End --- I second eapenabrm's suggestions. 1. Make sure the FPGA is programmed with the correct image. A simple way would be to connect a counter signal to an LED so it keeps blinking. 2. Check if lspci shows the Altera FPGA as a device ( On linux) or on the device manager ( on windows). Again check the device id in the PCIe QSYS component or what is returned by lspci are the same and make sure your driver is looking for the same device. 3. For the reads, I had made a mistake in connecting my QSYS components. The slave attached to the PCIe component did not return a readdatavalid, whereas the PCIe component expected it. Thus, I never "read" anything back. After I changed it, the reads started to work. 4. I would also suggest using a signal tap to monitor the BAR (register interface). You can trigger on reads or writes, while the driver is running. You can then see if the behaviour of the system is as expected on not. I used the following steps to setup the driver, and not their scripts rmmod altera_dma.ko rf /dev/altera_dma cat /proc/devices # choose a device number that isn't already taken. In this case I have used 243 mknod /dev/altera_dma c 243 0 chmod 0644 /dev/altera_dma ls -al /dev/altera_dma insmod altera_dma.ko rm -f ./user/user gcc ./user/user.c -o ./user/user ./run

Hi eapenabrm, Thank you for the reply! Sorry for having the following newbie questions :p: Is there any other way to check if the FPGA has been programmed successfully besides seeing the "Programmer" in Quartus Prime shows "100% (successful)" in the progress bar? When you said "check if the board has been enumerated properly in the PC", I am assuming you mean I should execute lscpi | grep Altera and see if the Altera device appears, right? If so, when the FPGA is not programmed with the reference design, the above command shows nothing. On the other hand, if the FPGA is programmed, I can see the following output 01:00.0 Non-VGA unclassified device: Altera Corporation Device e003 (rev ff) Does this mean that it is enumerated? May I ask how to check if the device ID matches with the one in the board? Is the "e003" in the above output the device ID the OS sees? Then how should I check whether the IP programmed on board has the same device ID? Using lsmod, I can see the following (in the cases where FPGA is programmed, lspci shows the Altera device, but /dev/altera_dma does not exist) Module Size Used by altera_dma 20480 0 or (in the case where FPGA is programmed, lspci shows the Altera device, and /dev/altera_dma exist) Module Size Used by altera_dma 20480 2 some other clarification about how i got the fpga programmed I am using the "two-machine" method, as stated on page 13 in this document (https://www.altera.com/content/dam/altera-www/global/en_us/pdfs/literature/an/an456.pdf), to program the FPGA. However, there was only once I could get the "./install" and the "./run" of the ref design software to see the device /dev/altera_dma. In all the other attempts, I never be able to make /dev/altera_dma to show up again (meaning I will see "chmod: cannot access /dev/altera_dma: No such file or directory" when executing "sudo ./install" of the ref design software) dmesg shows the following relevant messages, but I am not sure what went wrong: altera_dma: module verification failed: signature and/or required key missing - tainting kernel Altera DMA: altera_dma_init(), Nov 7 2017 10:46:02 Altera DMA 0000:01:00.0: Refused to change power state, currently in D3 Altera DMA 0000:01:00.0: can't enable device: BAR 0 not claimed Altera DMA 0000:01:00.0: pci_enable_device() failed Altera DMA 0000:01:00.0: goto err_enable Altera DMA 0000:01:00.0: goto err_initchrdev Altera DMA 0000:01:00.0: goto err_bk_alloc Altera DMA: probe of 0000:01:00.0 failed with error -22 Altera DMA: altera_dma_init(), Nov 7 2017 10:46:05 Altera DMA 0000:01:00.0: Refused to change power state, currently in D3 Altera DMA 0000:01:00.0: can't enable device: BAR 0 not claimed Altera DMA 0000:01:00.0: pci_enable_device() failed Altera DMA 0000:01:00.0: goto err_enable Altera DMA 0000:01:00.0: goto err_initchrdev Altera DMA 0000:01:00.0: goto err_bk_alloc Altera DMA: probe of 0000:01:00.0 failed with error -22 Any suggestion is much appreciated!!!!

altera_dma driver for PCIe | Altera Community

34 Replies

Altera_Forum
Honored Contributor
8 years ago
Thank you very much for reply, Below my log.

I am doing the following step

./run
start DMA 1
Loop 8

My Problem I don't get read passed /write passed 1 , with very low throughput less 0.1G

[ 1316.143821] Altera DMA: altera_dma_remove(), __DATE__ __TIME__
[ 1316.161570] Altera DMA: altera_dma_init(), __DATE__ __TIME__
[ 1316.161596] Altera DMA 0000:02:00.0: pci_enable_device() successful
[ 1316.161643] Altera DMA 0000:02:00.0: pci_enable_msi() successful
[ 1316.161650] Altera DMA 0000:02:00.0: using a 64-bit irq mask
[ 1316.161651] Altera DMA 0000:02:00.0: irq pin: 1
[ 1316.161652] Altera DMA 0000:02:00.0: irq line: 10
[ 1316.161653] Altera DMA 0000:02:00.0: irq: 43
[ 1316.161654] Altera DMA 0000:02:00.0: request irq: 10
[ 1316.161655] Altera DMA 0000:02:00.0: BAR[0] 0xd2208000-0xd22081ff flags 0x0014220c, length 512
[ 1316.161657] Altera DMA 0000:02:00.0: BAR[1] 0x00000000-0x00000000 flags 0x00000000, length 0
[ 1316.161658] Altera DMA 0000:02:00.0: BAR[2] 0x00000000-0x00000000 flags 0x00000000, length 0
[ 1316.161659] Altera DMA 0000:02:00.0: BAR[3] 0x00000000-0x00000000 flags 0x00000000, length 0
[ 1316.161660] Altera DMA 0000:02:00.0: BAR[4] 0xd2200000-0xd2207fff flags 0x0014220c, length 32768
[ 1316.161661] Altera DMA 0000:02:00.0: BAR[5] 0x00000000-0x00000000 flags 0x00000000, length 0
[ 1316.161672] Altera DMA 0000:02:00.0: BAR[0] mapped to 0xffffc90000fe2000, length 512
[ 1316.161677] Altera DMA 0000:02:00.0: BAR[4] mapped to 0xffffc90001540000, length 32768
[ 1318.795184] Read DMA times out
[ 1318.795186] DWORD = 00000200
[ 1318.795187] Desc = 00000080
[ 1318.992722] Write DMA times out
[ 1318.992722] DWORD = 00000200
[ 1318.992723] Desc = 00000080
[ 1319.211383] Simultaneous DMA times out
[ 1319.211383] DWORD = 00000200
[ 1319.211384] Desc = 00000080
[ 1323.836579] Read DMA times out
[ 1323.836581] DWORD = 00000200
[ 1323.836581] Desc = 00000080
[ 1324.033586] Write DMA times out
[ 1324.033586] DWORD = 00000200
[ 1324.033587] Desc = 00000080
[ 1324.251871] Simultaneous DMA times out
[ 1324.251871] DWORD = 00000200
[ 1324.251872] Desc = 00000080
[ 1332.790695] Read DMA times out
[ 1332.790697] DWORD = 00000200
[ 1332.790698] Desc = 00000080
[ 1332.988569] Write DMA times out
[ 1332.988570] DWORD = 00000200
[ 1332.988571] Desc = 00000080
[ 1333.207756] Simultaneous DMA times out
[ 1333.207757] DWORD = 00000200
Altera_Forum
Honored Contributor
8 years ago
TO_BE_DONE
Altera_Forum
Honored Contributor
8 years ago
Hello,

I don't know if anyone has the same issue, PCIe linux driver can see PCIe Gen1, however it can't see Gen2 and Gen3 (when I run lspci -vvv, I get unknown header 7f)
To get stratix v PCIe Gen3 working is so important to my research, so I would appreciate any help!

Rasha
Altera_Forum
Honored Contributor
8 years ago
Hello,

I don't know if you had same issue like mine, PCIe linux driver can see PCIe Gen1, however it can't see Gen2 and Gen3 (when I run lspci -vvv, I get unknown header 7f).
Is yours working on Gen3?
Also, when I use Gen1, I still don't get read /write passed working, although Bar0 and Bar4 mapped correctly!

To get stratix v PCIe Gen3 working is so important to my research, so I would appreciate any help!
Altera_Forum
Honored Contributor
8 years ago
Hi, talking about reference design for Arria V, i solved the issue modifying PCIe memory address space from 32(default in reference design) to 64, maybe will work for you also. You just need call Qsys and there, click in avmmdma_pcie instance. In the configuration window you will see the memory address space parameter.
Altera_Forum
Honored Contributor
8 years ago
This thread totally slipped my mind. Sincere apology for the super delayed reply. But also I have been pretty hesitant to post my solution because I think my problem was just caused by my ignorance about how PCIe device and the IP works.

But, anyways, there were two reasons why it didn't work for me:
(1) I was using a PCIe extension cable to connect the device
(2) I didn't successfully recompile the reference design with the version of Quartus I use

After I plug the FPGA directly to the PCIe slot, regenerate (update) the Qsys file, and successfully recompile the entire design with the new version of Quartus, I was able to run the reference design with the speed of 5.X GB/s or up.

Hope this helps.
Altera_Forum
Honored Contributor
8 years ago
Thanks for your reply. I don't think it helps since my memory address is already 64. Also, I am using DE5 net Stratix V
Altera_Forum
Honored Contributor
8 years ago
Thanks for your reply. Are you using stratix V DE5 net ??
Altera_Forum
Honored Contributor
8 years ago
No, i used an Arria V Starter Kit.
Altera_Forum
Honored Contributor
8 years ago
The low throughput numbers that everyone is seeing in DMA module is due to DMA timeout happening.
DMA timeout is occurring because the Descriptor controller FIFO addresses for Read and Write in altera_dma.h do not
match the values in FPGA (reported in QSYS).
The base addresses of wr_dts_slave and rd_dts_slave in QSYS and the defines WR_CTRL_BUF_BASE_* and RD_CTRL_BUF_BASE_* in altera_dma.h should match.

So basically if your addresses in FPGA are:
rd_dts_slave - 0x1_0000 (Base)
wr_dts_slave - 0x1_2000 (Base)

Then altera_dma.h should define:# define RD_CTRL_BUF_BASE_LOW 0x00010000# define RD_CTRL_BUF_BASE_HI 0x00000000# define WR_CTRL_BUF_BASE_LOW 0x00012000# define WR_CTRL_BUF_BASE_HI 0x00000000

Forum Discussion

altera_dma driver for PCIe

34 Replies

Recent Discussions

About Design Limitations and Known Issues

About Dual Simplex for Agilex 3

Where is High Speed Transceiver Demo Design in FPGA Wiki ?

Can't find Agilex 7 M I/O PLL Reconfiguration Design Example

Enabling DFE Adaptation on Cyclone 10 GX