Direct GPU-FPGA DMA

Hello,

I am trying to implement direct DMA communication between a Nvidia Quadro and a Stratix V board over PCIe. Nvidia supports this with its GPUDirect RDMA technology. I have read through the code of Altera's PCIe driver and as far as I can tell I only need to get the physical address of the GPU memory and send it to the DMA controller, is that correct?

I already succeeded to transfer data from the FPGA to the GPU that way, however when I try the other direction (by swapping read and write addresses), the whole system freezes.

My kernel module code looks something like this (simplified & error checking omitted):

//pin gpu memory and get the physical address
nvidia_p2p_get_pages(0, 0, virtual_gpu_address, size, &page_table, free_callback, NULL);
unsigned long physical_gpu_address=page_table->pages->physical_address;
//set up a dma descriptor
unsigned int att_row=0;
size_t pcietxs_addr = (size_t) ACL_PCIE_TX_PORT 
                     | (att_row * ACL_PCIE_DMA_MAX_ATT_PAGE_SIZE) 
                     | (physical_gpu_address & (ACL_PCIE_DMA_MAX_ATT_PAGE_SIZE-1));
struct DMA_DESCRIPTOR dmadesc;
//direction from FPGA to GPU (works)
//dmadesc.read_address=    LO32(fpga_address);
//dmadesc.read_address_hi= HI32(fpga_address);
//dmadesc.write_address=   LO32(pcietxs_addr);
//dmadesc.write_address_hi=HI32(pcietxs_addr);
//direction from GPU to FPGA (crashes)
dmadesc.read_address=    LO32(pcietxs_addr);
dmadesc.read_address_hi= HI32(pcietxs_addr);
dmadesc.write_address=   LO32(fpga_address);
dmadesc.write_address_hi=HI32(fpga_address);
dmadesc.bytes = size;
dmadesc.burst = 0;
dmadesc.stride = 0x00010001;
dmadesc.control = (unsigned int)( ACL_PCIE_GET_BIT(DMA_DC_GO) 
                |  ACL_PCIE_GET_BIT(DMA_DC_EARLY_DONE_ENABLE)
                |  ACL_PCIE_GET_BIT(DMA_DC_TRANSFER_COMPLETE_IRQ_MASK)
                   );
//send data to the DMA controller
//functions from aclpci_dma.c
set_att_entry(aclpci, physical_gpu_address, att_row);
dma_desc_write (aclpci, 0, &dmadesc);

Does anyone have an idea what could be wrong?

Thanks,

Alexander