Forum Discussion
Honored ContributorAltera SGDMA & TSE
Hi all,
I am working on TSE and is using sgdma for avalon_stream_to_mem descriptor, now in my code I have four descriptors,I am receiving network packets in my local buffer from sgdma in every sgdma callback function, now the issue is that I am getting lower throughput and thus lowering my transfer speed. Can any body know how to use sgdma at its best for getting higher data rate?20 Replies
Altera_ForumAre your four descriptors chained? When is your callback called? Is it when the chain is completed, or for each descriptor completion?
To make most use of the SGDMA the secret is to be sure it is always kept busy, and that is always has at least one usable descriptor. If you wait until it has parked at the end of the chain before you set a new one then you are loosing some time. I've never used the callback mechanism but I've used my own ISR, but I think this is possible with callbacks too. Often the problem with low throughput isn't the SGDMA itself but the software that controls it. Be sure that you do as little processing as possible when handling the SGDMa interrupts, and especially use pre-allocated memory buffers. A malloc() call is very costly. Be sure also that your processing of the packet data isn't the bottleneck. Obviously it should last less time than the interval you have between two packets.- Altera_Forum
Hi..
My descriptor chain size is 1, and callback is called when chain is completed. I am using Altera Sgdma drivers and I have tried but couldn't get the most of sgdma. In every call back I am getting a single frame from received in TSE. I have cleaned all the logic from the callback function & no malloc or memcpy is done and also using pre allocated buffer. So now what should I do to get the higher throughput. With current config i am getting my data transferred at upto 270KBPS but I need this to MBPS. - Altera_Forum
You should definitely be able to have a higher sample rate than that. You could try a longer chain if you can handle several buffers at the same time. What Nios CPU are you using and at what frequency? Do you compile your code with optimization? (-O2)
Do you call the do_async_transfer() function as soon as possible in the callback? To get the highest throughput it is important to do it with the lowest delay possible. What your callback should do is first to look for a new buffer for the next transfer, set up the DMA for the async transfer, and only then process the received packet. That way the DMA can begin loading the next packet while you process the current one. If you have enough onchip RAM in your FPGA it can also be a good idea to use it for your network packets instead of main RAM. You can even use a dual port one with one port connected to the DMA and the other one to the NIOS CPU. - Altera_Forum
Hi..I am using NIOS II Small CPU at 125MHz frequency & ram size is 128MB. I have tried with -O2 optimization but didn't see any difference in performance. In the callback function first I am calling tse_mac_rcv() and then after checking for chain complete mask & status tse_mac_axRead() is called which inside calls do_async_transfer(). After that I am putting my logic for received packet. Is this the correct way to handle ISR callback? With this configuration my ISR callback is called max upto 250-300 times in a second and which indicates the maximum speed we can achieve to 250-300KBPS. Is there any other way we can handle these logic? I have not tried with multiple chain desciptor.
- deepag
New Contributor
Hi..
Am using Quartus version 21.1 for FPGA programming and software development for NIOS II SDK Tool. Am working on TSE ethernet. we have added TSE IP in Quartus and BSP files got generated to NIOS Tool , but i could not find these functions like tse_mac_rcv() , tse_mac_axRead() , tse_mac_raw_send().
can you please send the files which consists of these functions .
Thanks & Regards
Deepa G
- Altera_Forum
It looks okay but I haven't used the tse_mac_* functions in a while and I don't remember how bloated they are. You could have a look inside them and see what they do. Ideally they should just signal the IP stack that a new packet has arrived and return, instead of doing all the packet processing.
What network protocol are you using? TCP is slower than UDP as it requires more processing from the IP stack. How big is the instruction cache on the Nios? Does changing to a /f core improve anything? - Altera_Forum
These functions I have taken from niche stack for ucosii. I have removed all the OS dependent code and modified it to use with bare metal software. So all these functions internally calls do_async_transfer and other basic APIs. My actual job is to take raw ethernet frames from USB port and send all those to TSE and similarly all the packets received from TSE to USB. In this case I am receiving large number of raw frames in a single usb transfer,so I segregate them all and send one by one to TSE using tse_mac_raw_send() api. In the Sgdma callback I am receiving single packet/raw frame and send those to USB. The speeds I am getting tested with iperf 4.72MBPS from USB to TSE and 300-315KBPS from TSE to USB. Now I want to increase the speed from TSE to USB part. Can you just tell me what can be the possible bottleneck in the system? And yes NIOS II instruction cache size is 32k.
- Altera_Forum
Can you use a profiler? It should indicate what part of your code is using the most CPU and should help you find what to optimize.
Another way to do that is to comment away some part of the functionality and see if you get any signoficate speed increase. If yes it means the bottleneck is in the commented code. Do you also concatenate together several ethernet frames before sending them on USB? If not and if there is a significant overhead when sending through USB it could be it. How are you sending to USB? Are you using a DMA that reads from the same buffer the ethernet frame was received in? If your software copies data around then it can reduce the bandwidth significantly. - Altera_Forum
No at the moment I am not concatenating ethernet frames before sending them to usb. I tried once but I found that I have to wait for frames to come then I concatenate and also I have to wrap all the frames with USB header and trailer and attach all the information like length of each frame, number of frames attached etc. I am using NCM class for doing this so I have to follow NCM fixed format for sending data. And so these all takes longer and results in increased delay if i ping from remote pc. And also I am not using DMA because each frame sizes are almost 60,90 and maximum 1514 bytes. Do you really think that concatenating number of frames will increase in speed?
- Altera_Forum
Without profiling it's not possible to give a certain answer but yes I think so. Packing lots of small packets with software takes some time and will reduce the bandwith. If you want haigh bandwidth while keeping the same solution you need to pack several packets together. With the SGDMA you can even configure the DMA to automatically pick the different fragments where they are in memory and assemble them, reducing the CPU usage and again increasing the bandwidth. But as you say it will increase the latency. You have to choose between latency and bandwidth.
Another solution to have both a high bandwidth and a low latency would be to to the whole Ethernet to USB conversion in hardware instead, and don't use at all the Nios CPU or the DMAs. But it's more work. - Altera_Forum
Hi,
I tried accumulating multiple raw frames and wrap'em with usb header and trailer, but it adds latency while ping, one more issue is that my USB can't cope up with the speed with received frames frequency as from the beginning Rx interrupt hits very fast and so my queue gets full so data packets are dropped. Can you please tell me that how to handle this kind of situation when your up flow is polling based and down flow is interrupt based? And how these stacks like lwip or nichestack handles data as in all of them reception is sgdma interrupt based?
