Has anyone used the Stride feature (available with the extended descriptor)? I using the Qsys PCIe example design on the wiki. When I enable the extended descriptor and stride option, the data transfer gets corrupted and the PCIe bombs out. I know you have to write to a different address to set the GO bit, and have to write to a register to set the stride length. I basically just want to do large transfers to a custom component with a 256 bit interface. I have an SR open with Altera, but they are slower than molasses.

Striding across PCIe doesn't sound like a good idea to me. For starters when stride is enabled you have to disable bursting which will kill your PCIe performance. Also related to that is that for each word transfer a new PCIe packet will need to be formed because stride will cause addresses to *not* be continous. Perhaps your impression of the feature is different than what is implemented. For example if you setup a stride of 16 for a DMA that is 64-bits wide these are the addresses you should see accessed assuming a start location of 0x0: 0x000-0x008 0x080-0x088 0x100-0x108 etc....

I just want to do a continuous read, and want the write address to roll over after so many bytes or reset back to the base address. As an example, I want to transfer 2 MByte of data, to a 256 bit wide FIFO. I understand I could probably just chain up many small descriptors and increment the address in software, but why should I have to do that, the DMA engine should be capable of handling this. When the stride feature is enabled, the system always ends up in an irrecoverable state and the only way to fix that is to reprogram the FPGA and restart the PC.

What you describbed can be accomplisted with a MM-->ST mSGDMA (i.e. no write master) and if your FIFO is wider than the data path then a data format adapter will convert it over to the correct width. For example if the PCIe slave width is 64-bit with a burst length of 32 I would use the mSGDMA in MM-->ST mode with a 64-bit data path, max burst length of 32 and then use a 1:4 data format adapter to take four 64-bit beats and cram them into the 256-bit FIFO. Of course you can do this with a memory mapped FIFO but it would take some additional logic to funnel multiple addresses into the same word before pushing it into the FIFO. Most of the time users will create a FIFO with a really large address span and alias the addresses internally to push data in. I personally never use MM for FIFOs unless absolutely necessary and in your case I don't think that's the case at all. Avalon-ST exists for stuff like this so I would simplify things and use the most appropriate interfaces to get your job done instead of using MM for things it's not really meant for.

Hi BadOmen, I have the same Problem: I need two "as wide as can be"-FiFos for some outside-of-QSYS-Logic (one to write and one to read). I also took the MM-Solution, but your way seems to be the better one. I tried to implement it somehow but without success. Can you explain it for a dummy like me, which IP I have to implement and what has to be configured? Thanks Steffen

I recommend using a pair of DMAs, one setup for MM-->ST and the other for ST-->MM. This means one DMA is capable of only reads and the other is only capable of writes. Since the master blocks contain a FIFO internally it would probably be sufficient to export the streaming port of each master block so that your external logic can access the FIFO directly. For the read DMA you would configure the dispatcher for MM to ST mode and connect the read master to it. For the write DMA you would configure the dispatcher to ST to MM mode and connect the write master to it. Also I recommend visiting the mSGDMA wiki page since I uploaded an update to the read master to fix a FIFO overflow bug.

mSGDMA stride reads/writes | Altera Community

27 Replies

Altera_Forum
Honored Contributor
11 years ago
I don't see anything wrong. I would probably disable the mSGDMA global interrupt mask before unregistering it's interrupt in destruct_sgdma.

It appears you are trying to perform a ST-->MM transfer using flow control so are you sure the transfer is completing? The transfer according to the descriptor stops when EOP arrives to the write master and that last piece of data is written out to memory.

I noticed that you are passing this in as the maximum transfer length when creating the descriptor: 4*NCELL. That has to be specified in bytes if you are not already doing so. Also if that amount of data or more arrives at the write master before the EOP beat comes into the ST port of the write master then it will terminate the transfer earlier. Note that the early termination may happen a few bytes more than 4*NCELL if you have unaligned/aligned transfers enabled in the write master hardware.

Note if you don't want to be using ST packets at all then I wouldn't pass in 'DESCRIPTOR_CONTROL_END_ON_EOP_MASK' into the descriptor forming API.

Altera_Forum

Honored Contributor

11 years ago

Hi BadOmen... It's me again...

I do feed signal generator to ADC -> then processed by FFT -> read by mSGDMA ST-MM with nios task (double buffered)

void ReadStreamTask() {
	INT8U err;
	reset_dispatcher(MSGDMA_RX_CSR_BASE);
	alt_ic_isr_register(MSGDMA_RX_CSR_IRQ_INTERRUPT_CONTROLLER_ID,
			MSGDMA_RX_CSR_IRQ, msgdma_rx_complete_isr, NULL, NULL);
	enable_global_interrupt_mask(MSGDMA_RX_CSR_BASE);
	Semaphore = OSSemCreate(1);
	Semaphore = OSSemCreate(2);
	while (1) {
		if(msgdma_int_rx == 0) {
			msgdma_int_rx = 1;
			do{
				storage_ptr = (storage_ptr + 1) % 2;
				OSSemPend(Semaphore, 2, &err);
			}while(err == OS_ERR_TIMEOUT);
			IOWR_ALTERA_AVALON_PIO_DATA(PERIPHERAL_SUBSYSTEM_LED_PIO_BASE, 0xff - storage_ptr - 1);
			construct_standard_st_to_mm_descriptor(desc_rx_ptr, SGDMAMemory, 4*NCELL,
					DESCRIPTOR_CONTROL_TRANSFER_COMPLETE_IRQ_MASK | DESCRIPTOR_CONTROL_END_ON_EOP_MASK);
			write_standard_descriptor(MSGDMA_RX_CSR_BASE, MSGDMA_RX_DESCRIPTOR_SLAVE_BASE, desc_rx_ptr);
		}
		else if(msgdma_int_rx == 2) {
			msgdma_int_rx = 0;
			int i;
			err = OSSemPost(Semaphore);
			OSTimeDlyHMSM(0, 0, 0, 5);
		}
	}
	/*
	 * Should not be here as long as while = 1
	 */
	destruct_sgdma();
}

After transferred through ethernet, by simple TCP IP program to draw FFT graph. I've got only some partial data.

			do {
				read_ptr = (read_ptr + 1) % 2;
				OSSemPend(Semaphore, 2, &err);
			} while(err == OS_ERR_TIMEOUT);
//			printf("------------------------------------------\n%d\n------------------------------------------\n", sizeof(iNetworkRadarData.data));
//			printf("Send package 1\n");
			length = sizeof(SGDMAMemory)/2;
			error_code = send(conns->fd, (char*) &length, sizeof(length), 0);
			error_code = send(conns->fd, (char*) &SGDMAMemory, length, 0);
//			printf("Send package 2\n");
			error_code = send(conns->fd, (char*) &length, sizeof(length), 0);
			error_code = send(conns->fd, (char*) &SGDMAMemory + length, length, 0);
			construct_standard_mm_to_st_descriptor(desc_tx_ptr, SGDMAMemory, 4*512,
					DESCRIPTOR_CONTROL_GENERATE_SOP_MASK | DESCRIPTOR_CONTROL_GENERATE_EOP_MASK);
			write_standard_descriptor(MSGDMA_TX_CSR_BASE,
					MSGDMA_TX_DESCRIPTOR_SLAVE_BASE,
					desc_tx_ptr);
//			memset(SGDMAMemory, 0, sizeof(SGDMAMemory));
			err = OSSemPost(Semaphore);
			OSTimeDlyHMSM(0,0,0,5);

Please see the following picture

Thank you...

Altera_Forum
Honored Contributor
11 years ago
My question:
I couldn't get full transfer mSGDMA Memory with nios, Am I missed something?
Please help... thank you..
Altera_Forum
Honored Contributor
11 years ago
So you DMA data from the FFT into memory, then have another DMA transfer to push the recently copied memory locations into the MAC correct? If so have you debugged the first transfer to ensure that it's completing correctly because at least that will narrow down the culprit.

The setup looked ok to me except it doesn't look like you are using interrupts when you construct the descriptor. So there are two layers of interrupt setup that needs to happen, the global interrupts need to be enabled in the control register of the CSR port, but in order for interrupts to fire you have to enable them when constructing the descriptor as well. There are a few interrupt bits in the control field of the descriptor but the one you want is the transfer done interrupt.

Some additional info: The early termination interrupt flag is for ST-->MM packet transfers, and it fires (if enabled) when the DMA ends up moving more streaming data than what you specify as the transfer length. In other words the DMA lets you put a cap on the amount of data moved in a ST-->MM transfer in the event that the EOP bit never arrives. There is also an error interrupt as well to let you know if any of the ST-->MM data has any of the error bits asserted.

Altera_Forum

Honored Contributor

11 years ago

Hmmm.. yes you are right.. I use triple speed ethernet example and adds some module for my purpose.

And yes, I have debugged it.. I use mSGDMA MM-ST to check streaming data in Signal Tap, and works fine. But not in TCP IP

Yes, I have ISR Callback

/*
 * ISR Function when mSGDMA Interrupt happens
 */
static void msgdma_rx_complete_isr(void *context) {
//	IOWR_ALTERA_AVALON_PIO_DATA(PERIPHERAL_SUBSYSTEM_LED_PIO_BASE, 0xfd);
	stop_dispatcher(MSGDMA_RX_CSR_BASE);
	while(read_stopped(MSGDMA_RX_CSR_BASE) == 0);
	reset_dispatcher(MSGDMA_RX_CSR_BASE);
	enable_global_interrupt_mask(MSGDMA_RX_CSR_BASE);
	msgdma_int_rx = 2;
}

Thanks for your additional info..

In the latest, there are some things that I have changed. And it worked for me...

Before: Memory Altmemphy DDR

After: double Onchip Memory for successive Nios and mSGDMA

Before: In MM to TCP IP Task

			do {
				read_ptr = (read_ptr + 1) % 2;
				OSSemPend(Semaphore, 2, &err);
			} while(err == OS_ERR_TIMEOUT);
//			printf("------------------------------------------\n%d\n------------------------------------------\n", sizeof(iNetworkRadarData.data));
//			printf("Send package 1\n");
			length = sizeof(SGDMAMemory)/2;
			error_code = send(conns->fd, (char*) &length, sizeof(length), 0);
			error_code = send(conns->fd, (char*) &SGDMAMemory, length, 0);
//			printf("Send package 2\n");
			error_code = send(conns->fd, (char*) &length, sizeof(length), 0);
			error_code = send(conns->fd, (char*) &SGDMAMemory + length, length, 0);
			construct_standard_mm_to_st_descriptor(desc_tx_ptr, SGDMAMemory, 4*512,
					DESCRIPTOR_CONTROL_GENERATE_SOP_MASK | DESCRIPTOR_CONTROL_GENERATE_EOP_MASK);
			write_standard_descriptor(MSGDMA_TX_CSR_BASE,
					MSGDMA_TX_DESCRIPTOR_SLAVE_BASE,
					desc_tx_ptr);
//			memset(SGDMAMemory, 0, sizeof(SGDMAMemory));
			err = OSSemPost(Semaphore);
			OSTimeDlyHMSM(0,0,0,5);

After: In MM to TCP IP Task

			do {
				read_ptr = (read_ptr + 1) % 2;
				OSSemPend(Semaphore, 2, &err);
			} while(err == OS_ERR_TIMEOUT);
//			length = sizeof(SGDMAMemory)/2;
			alt_u32 memory;
			for(i = 0; i < 512; i++) {
				if(read_ptr == 0)
					memory = IORD_32DIRECT(SGDMA_MEM_0_BASE + i*sizeof(alt_u32), sizeof(alt_u32));
				else
					memory = IORD_32DIRECT(SGDMA_MEM_1_BASE + i*sizeof(alt_u32), sizeof(alt_u32));
			}
			length = 4*1024/8;
			for(i = 0; i <4; i++) {
				error_code = send(conns->fd, (char*) &length, sizeof(length), 0);
				error_code = send(conns->fd, (char*) memory + i*length, length, 0);
			}
			err = OSSemPost(Semaphore);
			OSTimeDlyHMSM(0,0,0,5);

Altera_Forum
Honored Contributor
11 years ago
Note: I do IORD_32DIRECT to copy mSGDMA memory...
when I failed with memcpy...
Altera_Forum
Honored Contributor
11 years ago
If you use IORD_32DIRECT or IOWR_32DIRECT you have to be careful of caching. For example if you use malloc to allocate a buffer and then access it using those cache bypassing macro you might have a coherency issue. Memory allocated from the heap might have already been cached and still persistent in the cache so if you perform a cache bypass to a location that is already cached then you'll have a coherency issue. To avoid this issue you should invalidate the cache for the address range that malloc returns to you to ensure you don't run into this corner case.

Forum Discussion

mSGDMA stride reads/writes

27 Replies

Recent Discussions

Avalon-ST configuration with Agilex 3 fails

Cyclone IV E – PLL Power Track Width Recommendation Clarification

Operating temperature for 10M08DCF256A7G

System PLL of Agliex5 PCIE example design cannot be locked after configuration

Download links not working