Linux mSGDMA Driver Agilex 5

I am trying to use stream-to-memory DMA to move data.

This is the design, showing the important Quartus connections: 

Which corresponds to this address config:

# address map
write_sdram       = 0x00000000
read              = NULL (has no meaning in streaming to MM mode)
msgdma_csr        = 0x22000040
msgdma_descriptor = 0x22000060

# msgdma offsets
msgdma_csr_control          = 0x4 + 0x22000040 = 0x22000044
msgdma_descriptor_write_low = 0x4 + 0x22000060 = 0x22000064
msgdma_descriptor_length    = 0x8 + 0x22000060 = 0x22000068
msgdma_descriptor_control   = 0xc + 0x22000060 = 0x2200006c

The oscillator IP generates a group of known numbers, so the readbacks change each time - but are predictable and so the hardware is verifiable.

In U-Boot; I have verified that this design. This script asks the DMA to make two transfers, and reads back the results each time.

SOCFPGA_AGILEX5 # mw 0x22000044 0x1 # Stop Dispatcher
SOCFPGA_AGILEX5 # mw 0x22000044 0x20 # Stop descriptors
SOCFPGA_AGILEX5 # mw 0x22000044 0x2 # Reset Dispatcher
SOCFPGA_AGILEX5 # mw 0x22000064 0x00000000 # Write address
SOCFPGA_AGILEX5 # mw 0x22000068 0x8 # Write length in bytes
SOCFPGA_AGILEX5 # md 0x22000040 1 # Check status
22000040: 00000002                             ....
SOCFPGA_AGILEX5 # mw 0x2200006c 0x02000000 # Wait for response
SOCFPGA_AGILEX5 # sleep 1
SOCFPGA_AGILEX5 # mw 0x2200006c 0x82000000 # execute
SOCFPGA_AGILEX5 # md 0x22000040 1 # Check status
22000040: 00000002                             ....
SOCFPGA_AGILEX5 # md 0x00000000 1 # read
00000000: 00000000                             ....
SOCFPGA_AGILEX5 # mw 0x22000044 0x2 # reset
SOCFPGA_AGILEX5 # mw 0x2200006c 0x82000000 # execute
SOCFPGA_AGILEX5 # md 0x00000000 1 # read
00000000: 67676767                             ....

When trying to do the same in linux using 'devmem2', I found that the readbacks did not change, so I decided to look for some examples.

I found this stream-to-mapped mSGDMA driver built for the Cyclone V, and so re-purposed it.

The driver example above both reads and writes to the device using the ‘dd’ utility in linux, but I am for now only interested in the read command. To do so I needed some information...

Based on the .qsys design file output of my project, the mSGDMA block generates this interrupt:

...
<altera:connection altera:kind="interrupt" altera:version="24.3" altera:start="subsys_hps.f2h_irq0_in" altera:end="msgdma_0.csr_irq">
  <altera:connection_parameter altera:parameter_name="irqNumber" altera:parameter_value="2">
  </altera:connection_parameter>
</altera:connection>
...

And according to the interrupt docs, the above csr interrupt value=2 maps to the GIC interrupt 51, which is 19 when referenced from the linux device tree.

With this information, I have defined the following mSGDMA element in the .dts:

...
msgdma_test: msgdma_test@22000000 {
compatible = "msgdma_test";
    	reg = <0x0 0x22000000 0x0 0x100>;
	interrupt-parent = <&intc>;
	interrupts = <0 19 4>;
};
...

Which only differs from the original in that it has a different interrupt number, and different base address - also accounting for the lwhps2fpga offset, due to it being the Agilex5 and not the Cyclone.

The driver code from the REDS blog post has also been edited in the following way:

1. I need to change a deprecated function in the main driver code: devm_ioremap_nocache to demv_ioremap.
2. I want to see the data which has been moved - to do so have added an info print to the msgdma_read method: printk(KERN_INFO "MSGDMA: Read buffer content: %.*s\n", (int)read_ret, (char*)data->dma_buf_rd);
3. There is only 1 interrupt, so we set data->msgdma1_irq = platform_get_irq(pdev, 0);

Since there is no ability coded to fabric to perform msgdma_write - we can also remove memory allocations, remaps and references to msgdma0 and other write related objects.

I think the following shows that the DMA is firing interrupts, and so I am confident that I have connected the correct interrupt:

cat /proc/interrupts | grep msgdma
50:          2          0          0          0     GICv3  51 Level     msgdma1

However, when looking at the kernel outputs after requesting reads via dd:

$ dd if=/dev/msgdma_test of=/dev/null bs=8 count=1
$ dmesg | tail
[ ... ] MSGDMA: Read buffer (hex): 00 00 00 00 00 00 00 00
[ ... ] MSGDMA: Read buffer (hex): 00 00 00 00 00 00 00 00

The driver code in question is included in msgdma.c and header_msgdma.c, attached to the post (This forum does not seem to like .h files...)

I was hoping someone might have an idea of where I went wrong!

Any helpers are much appreciated!

Thanks for your time,

K