Hi everyone, i'm using an Arria V starter kit for PCIe application and i used the reference design posted in altera wiki. The application software reported success in one PC but not in other with different features. The driver code is installed in two with success but in one, user application reported timeout for read, write and simultaneous RW. Had anyone has the same problem? Have anyone some suggestion about what can i do to try solve this problem?

Hi there! I am having the same issue with timeout being reported for read and write. Have you found a solution for it? This is the output I see. I am using the driver for linux. ********************************************** ** ALTERA AVMM DMA driver ** ** version 2.02 ** ** 1) start DMA ** ** 2) enable/disable read dma ** ** 3) enable/disable write dma ** ** 4) enable/disable simul dma ** ** 5) set num dwords (256 - 2048) ** ** 6) set num descriptors (1 - 128) ** ** 8) loop dma ** ** 10) exit ** ********************************************** Run Read ? 1 Run Write ? 1 Run Simultaneous ? 0 Read Passed ? 0 Write Passed ? 0 Simultaneous Passed ? 0 Read EPLast timeout ? 1 Write EPLast timeout ? 1 Number of Dwords/Desc : 512 Number of Descriptors : 128 Length of transfer : 256 KB Rootport address offset : 0 Read Time : 1 s and 366248 us Read Throughput : 0.000179 GB/S Write Time : 1 s and 361611 us Write Throughput : 0.000179 GB/S Simultaneous Time : 1 s and 362982 us Simultaneous Throughput : 0.000358 GB/S

I am encountering the same problem, in which my DMA read and writes always time out. ************************************************ ALTERA AVMM DMA driver ** ** version 2.02 ** ** 1) start DMA ** ** 2) enable/disable read dma ** ** 3) enable/disable write dma ** ** 4) enable/disable simul dma ** ** 5) set num dwords (256 - 2048) ** ** 6) set num descriptors (1 - 128) ** ** 8) loop dma ** ** 10) exit ** ********************************************** Run Read ? 1 Run Write ? 1 Run Simultaneous ? 1 Read Passed ? 0 Write Passed ? 0 Simultaneous Passed ? 0 Read EPLast timeout ? 1 Write EPLast timeout ? 1 Number of Dwords/Desc : 512 Number of Descriptors : 128 Length of transfer : 256 KB Rootport address offset : 0 Read Time : 1 s and 227400 us Read Throughput : 0.000199 GB/S Write Time : 1 s and 230367 us Write Throughput : 0.000198 GB/S Simultaneous Time : 1 s and 238158 us Simultaneous Throughput : 0.000394 GB/S When I tried the burst mode (loop dma), I got # DMA data error! Type in dmesg to show more details! And by checking dmesg I got the following Read DMA times out DWORD = 00000100 Desc = 00000080 Read DMA times out DWORD = 00000100 Desc = 00000080 Read DMA times out DWORD = 00000100 Desc = 00000080 Read DMA times out DWORD = 00000100 Desc = 00000080 Read DMA times out DWORD = 00000200 Desc = 00000080 Write DMA times out DWORD = 00000200 Desc = 00000080 Simultaneous DMA times out DWORD = 00000200 Desc = 00000080 command issued from user space doesn't exist 2608 command issued from user space doesn't exist 2608 Does anyone have any clue why this would happen? Any suggestion or hint is much appreciated!

Have you checked if the FPGA has been programmed successfully and if the board has been enumerated properly in the PC? If using Linux, run the lsmod command and check if the driver is loaded. Have you checked if the device ID in the driver matches with the one in the board?

--- Quote Start --- Have you checked if the FPGA has been programmed successfully and if the board has been enumerated properly in the PC? If using Linux, run the lsmod command and check if the driver is loaded. Have you checked if the device ID in the driver matches with the one in the board? --- Quote End --- I second eapenabrm's suggestions. 1. Make sure the FPGA is programmed with the correct image. A simple way would be to connect a counter signal to an LED so it keeps blinking. 2. Check if lspci shows the Altera FPGA as a device ( On linux) or on the device manager ( on windows). Again check the device id in the PCIe QSYS component or what is returned by lspci are the same and make sure your driver is looking for the same device. 3. For the reads, I had made a mistake in connecting my QSYS components. The slave attached to the PCIe component did not return a readdatavalid, whereas the PCIe component expected it. Thus, I never "read" anything back. After I changed it, the reads started to work. 4. I would also suggest using a signal tap to monitor the BAR (register interface). You can trigger on reads or writes, while the driver is running. You can then see if the behaviour of the system is as expected on not. I used the following steps to setup the driver, and not their scripts rmmod altera_dma.ko rf /dev/altera_dma cat /proc/devices # choose a device number that isn't already taken. In this case I have used 243 mknod /dev/altera_dma c 243 0 chmod 0644 /dev/altera_dma ls -al /dev/altera_dma insmod altera_dma.ko rm -f ./user/user gcc ./user/user.c -o ./user/user ./run

Hi eapenabrm, Thank you for the reply! Sorry for having the following newbie questions :p: Is there any other way to check if the FPGA has been programmed successfully besides seeing the "Programmer" in Quartus Prime shows "100% (successful)" in the progress bar? When you said "check if the board has been enumerated properly in the PC", I am assuming you mean I should execute lscpi | grep Altera and see if the Altera device appears, right? If so, when the FPGA is not programmed with the reference design, the above command shows nothing. On the other hand, if the FPGA is programmed, I can see the following output 01:00.0 Non-VGA unclassified device: Altera Corporation Device e003 (rev ff) Does this mean that it is enumerated? May I ask how to check if the device ID matches with the one in the board? Is the "e003" in the above output the device ID the OS sees? Then how should I check whether the IP programmed on board has the same device ID? Using lsmod, I can see the following (in the cases where FPGA is programmed, lspci shows the Altera device, but /dev/altera_dma does not exist) Module Size Used by altera_dma 20480 0 or (in the case where FPGA is programmed, lspci shows the Altera device, and /dev/altera_dma exist) Module Size Used by altera_dma 20480 2 some other clarification about how i got the fpga programmed I am using the "two-machine" method, as stated on page 13 in this document (https://www.altera.com/content/dam/altera-www/global/en_us/pdfs/literature/an/an456.pdf), to program the FPGA. However, there was only once I could get the "./install" and the "./run" of the ref design software to see the device /dev/altera_dma. In all the other attempts, I never be able to make /dev/altera_dma to show up again (meaning I will see "chmod: cannot access /dev/altera_dma: No such file or directory" when executing "sudo ./install" of the ref design software) dmesg shows the following relevant messages, but I am not sure what went wrong: altera_dma: module verification failed: signature and/or required key missing - tainting kernel Altera DMA: altera_dma_init(), Nov 7 2017 10:46:02 Altera DMA 0000:01:00.0: Refused to change power state, currently in D3 Altera DMA 0000:01:00.0: can't enable device: BAR 0 not claimed Altera DMA 0000:01:00.0: pci_enable_device() failed Altera DMA 0000:01:00.0: goto err_enable Altera DMA 0000:01:00.0: goto err_initchrdev Altera DMA 0000:01:00.0: goto err_bk_alloc Altera DMA: probe of 0000:01:00.0 failed with error -22 Altera DMA: altera_dma_init(), Nov 7 2017 10:46:05 Altera DMA 0000:01:00.0: Refused to change power state, currently in D3 Altera DMA 0000:01:00.0: can't enable device: BAR 0 not claimed Altera DMA 0000:01:00.0: pci_enable_device() failed Altera DMA 0000:01:00.0: goto err_enable Altera DMA 0000:01:00.0: goto err_initchrdev Altera DMA 0000:01:00.0: goto err_bk_alloc Altera DMA: probe of 0000:01:00.0 failed with error -22 Any suggestion is much appreciated!!!!

altera_dma driver for PCIe | Altera Community

34 Replies

Altera_Forum
Honored Contributor
8 years ago
Hi eapenabrm,

I am using the Arria 10 Dev Kit (https://www.altera.com/products/boards_and_kits/dev-kits/altera/kit-a10-gx-fpga.html)

Before seeing your suggestion, I was trying to use the reference design out of box so I hadn't tried to build the design. But I was sure that both the device side and the driver side specify the device id as e003. But as we've seen so far, it didn't work out of box.

After seeing your reply, I went into Qsys and saw the "Device Identification Registers" tab, and saw the "Device ID" register (which was set to e003). So I changed it to e093 (see image https://www.alteraforum.com/forum/attachment.php?attachmentid=14330 ), saved the qsys file, and generated the HDL with default options.

Unfortunately, when I tried to recompile this design with the new Qsys register setting to make this setting effective, the assembler, which seems to be the step generating the sof file, failed and complained the following:
Error (210027): Can't use configuration device EPCQL256 with selected programming mode

(See attachment https://www.alteraforum.com/forum/attachment.php?attachmentid=14331 )

I am not sure what I did wrong, but the only change was the Device ID register in the qsys file. I was using Quartus Prime Standard Edition 17.0. Could it be because of the version of Quartus? It seem that the Arria 10 reference design was built with 16.0.

Any further suggestion is much appreciated!

--- Quote Start ---
Which kit are you using the Arria V or the Arria 10 Dev Kit? When building the example design, did you notice the PCIe config space registers. They usually have these Vendor ID and DEvice ID registers in it, with the Vendor ID preset to the Altera one and Device ID being 00000 . If this is the case with your design, then you may need to set the appropriate DEvice ID in your PCIe component (E0073 for Arria V / E0093 for Arria 10/ or E001 for default) and build the system again. Program the FPGA on the board. Then in the driver code make sure you use the same Device ID that is in the HW and build the driver again. This way the Device ID will match. Once the Device is enumerated successfully, the BAR registers will get enabled and you should be able to read and write from them, perform DMA etc.
--- Quote End ---
multiple-attachments.zip36 KB

Altera_Forum

Honored Contributor

8 years ago

--- Quote Start ---

Which kit are you using the Arria V or the Arria 10 Dev Kit? When building the example design, did you notice the PCIe config space registers. They usually have these Vendor ID and DEvice ID registers in it, with the Vendor ID preset to the Altera one and Device ID being 00000 . If this is the case with your design, then you may need to set the appropriate DEvice ID in your PCIe component (E0073 for Arria V / E0093 for Arria 10/ or E001 for default) and build the system again. Program the FPGA on the board. Then in the driver code make sure you use the same Device ID that is in the HW and build the driver again. This way the Device ID will match. Once the Device is enumerated successfully, the BAR registers will get enabled and you should be able to read and write from them, perform DMA etc.

--- Quote End ---

Hi eapenabrm,

I just used Quartus Prime Standard Edition 16.0 successfully (1) change the device ID in the qsys file to e093, (2) generate HDL, (3) recompile the entire project. A new sof file has been successfully generated.

Then I program it to my Arria 10 FPGA, I can verify the change was effective by


$ sudo lspci -vv | grep Altera -A 50
01:00.0 Non-VGA unclassified device: Altera Corporation Device e093 (rev 01)
    Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
    Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
    Latency: 0
    Interrupt: pin A routed to IRQ 11
    Region 0: Memory at d0020000 (64-bit, prefetchable) 
    Region 4: Memory at d0000000 (64-bit, prefetchable)

Unfortunately, ./install and ./run still report the same error (that /dev/altera_dma is not found).

One good(?) news is that, checking dmesg, it seems to show fewer and slightly different error messages this time:


 altera_dma: module verification failed: signature and/or  required key missing - tainting kernel
 Altera DMA: altera_dma_init(), Nov  9 2017 19:16:36 
 Altera DMA 0000:01:00.0: init_chrdev() failed
 Altera DMA 0000:01:00.0: goto err_initchrdev
 Altera DMA 0000:01:00.0: goto err_bk_alloc
 Altera DMA: probe of 0000:01:00.0 failed with error -1
 Altera DMA: altera_dma_init(), Nov  9 2017 19:16:44 
 Altera DMA 0000:01:00.0: init_chrdev() failed
 Altera DMA 0000:01:00.0: goto err_initchrdev
 Altera DMA 0000:01:00.0: goto err_bk_alloc
 Altera DMA: probe of 0000:01:00.0 failed with error -1

compared to the following, which are the error messages seen in most of the previous attempts:


 altera_dma: module verification failed: signature and/or required key missing - tainting kernel Altera DMA: altera_dma_init(), Nov  7 2017 10:46:02 
 Altera DMA 0000:01:00.0: Refused to change power state, currently in D3
 Altera DMA 0000:01:00.0: can't enable device: BAR 0  not claimed
 Altera DMA 0000:01:00.0: pci_enable_device() failed
 Altera DMA 0000:01:00.0: goto err_enable
 Altera DMA 0000:01:00.0: goto err_initchrdev
 Altera DMA 0000:01:00.0: goto err_bk_alloc
 Altera DMA: probe of 0000:01:00.0 failed with error -22
 Altera DMA: altera_dma_init(), Nov  7 2017 10:46:05 
 Altera DMA 0000:01:00.0: Refused to change power state, currently in D3
 Altera DMA 0000:01:00.0: can't enable device: BAR 0  not claimed
 Altera DMA 0000:01:00.0: pci_enable_device() failed
 Altera DMA 0000:01:00.0: goto err_enable
 Altera DMA 0000:01:00.0: goto err_initchrdev
 Altera DMA 0000:01:00.0: goto err_bk_alloc
 Altera DMA: probe of 0000:01:00.0 failed with error -22

It seems that the function pci_enable_device() now successfully executed(?) and BAR 0 is successfully claimed(??)

But the init_chrdev() function still fails. :(((

Seems like I am making progress on the correct direction??

Altera_Forum
Honored Contributor
8 years ago
There are 2 versions of the Arria 10 Kit:

1. The Golden Systems Top Production Kit
2. The Golden Systems ES Kit.

Both these kits have different HW setup. You need to download and install the corresponding Installer packages from the site.

Let me look into this further. May take a while for me to reply.
Altera_Forum
Honored Contributor
8 years ago
Hi eapenabrm and divyaganti,

Good news! I just got it working! Thank you so much for your time and the help!
But I haven't figured out what was the root cause of the problems. I'll post an update/summary about my findings asap.

Really appreciate all your help!!!
Altera_Forum
Honored Contributor
8 years ago
Hi,

Good to know that you got it working. Can you post what exactly you did for it to work so that it could be a reference to others with the same issue on the forum.

-Abr
Altera_Forum
Honored Contributor
8 years ago
If using Linux, run the lsmod command and check if the driver is loaded.
Altera_Forum
Honored Contributor
8 years ago
I am using DE5-net Stratix V and I have same problems in getting throughput too low and dma data error! when I run pcie linux driver. In my case, I have pcie_enable successfully but I see BAR[4] which is responsible to provide the memory address to the control register all start , end , flags , and length zeros (in dmesg) and also read/write dma timeout. Any help, please?
Altera_Forum
Honored Contributor
8 years ago
Please, have you figure out the major cause for your problems?, I have same problem and I am interested to know your results. Thanks
Altera_Forum
Honored Contributor
8 years ago
Hi Rasha,

Can you post the run logs / messages from the linux commands here.
Altera_Forum
Honored Contributor
8 years ago
Thank you very much for reply, Below my log.

I am doing the following step

./run
start DMA 1
Loop 8

My Problem I don't get read passed /write passed 1 , with very low throughput less 0.1G

[ 1316.143821] Altera DMA: altera_dma_remove(), __DATE__ __TIME__
[ 1316.161570] Altera DMA: altera_dma_init(), __DATE__ __TIME__
[ 1316.161596] Altera DMA 0000:02:00.0: pci_enable_device() successful
[ 1316.161643] Altera DMA 0000:02:00.0: pci_enable_msi() successful
[ 1316.161650] Altera DMA 0000:02:00.0: using a 64-bit irq mask
[ 1316.161651] Altera DMA 0000:02:00.0: irq pin: 1
[ 1316.161652] Altera DMA 0000:02:00.0: irq line: 10
[ 1316.161653] Altera DMA 0000:02:00.0: irq: 43
[ 1316.161654] Altera DMA 0000:02:00.0: request irq: 10
[ 1316.161655] Altera DMA 0000:02:00.0: BAR[0] 0xd2208000-0xd22081ff flags 0x0014220c, length 512
[ 1316.161657] Altera DMA 0000:02:00.0: BAR[1] 0x00000000-0x00000000 flags 0x00000000, length 0
[ 1316.161658] Altera DMA 0000:02:00.0: BAR[2] 0x00000000-0x00000000 flags 0x00000000, length 0
[ 1316.161659] Altera DMA 0000:02:00.0: BAR[3] 0x00000000-0x00000000 flags 0x00000000, length 0
[ 1316.161660] Altera DMA 0000:02:00.0: BAR[4] 0xd2200000-0xd2207fff flags 0x0014220c, length 32768
[ 1316.161661] Altera DMA 0000:02:00.0: BAR[5] 0x00000000-0x00000000 flags 0x00000000, length 0
[ 1316.161672] Altera DMA 0000:02:00.0: BAR[0] mapped to 0xffffc90000fe2000, length 512
[ 1316.161677] Altera DMA 0000:02:00.0: BAR[4] mapped to 0xffffc90001540000, length 32768
[ 1318.795184] Read DMA times out
[ 1318.795186] DWORD = 00000200
[ 1318.795187] Desc = 00000080
[ 1318.992722] Write DMA times out
[ 1318.992722] DWORD = 00000200
[ 1318.992723] Desc = 00000080
[ 1319.211383] Simultaneous DMA times out
[ 1319.211383] DWORD = 00000200
[ 1319.211384] Desc = 00000080
[ 1323.836579] Read DMA times out
[ 1323.836581] DWORD = 00000200
[ 1323.836581] Desc = 00000080
[ 1324.033586] Write DMA times out
[ 1324.033586] DWORD = 00000200
[ 1324.033587] Desc = 00000080
[ 1324.251871] Simultaneous DMA times out
[ 1324.251871] DWORD = 00000200
[ 1324.251872] Desc = 00000080
[ 1332.790695] Read DMA times out
[ 1332.790697] DWORD = 00000200
[ 1332.790698] Desc = 00000080
[ 1332.988569] Write DMA times out
[ 1332.988570] DWORD = 00000200
[ 1332.988571] Desc = 00000080
[ 1333.207756] Simultaneous DMA times out
[ 1333.207757] DWORD = 00000200

Forum Discussion

altera_dma driver for PCIe

34 Replies

Recent Discussions

Need a AMI Ibis for GXB on a 10AX115N3

Agilex-5 supported Transceivers

Looking for example testbench for Cyclone-V Transceiver Custom_PHY IP from UG-01080-1.7

Agilex™ 7 I-Series FPGA Transceiver SoC Development Kit GSRD Build

Tranceiver Enhanced PCS Basic mode questions