Forum Discussion
Altera_Forum
Honored Contributor
Honored ContributorEthernet performance using TSE in uClinux
Hi all,
I am measuring ethernet performance on a NIOS II based board with an Altera TSE IP Core running uClinux, and it's been disappointing so far. I've done timing using TCP and UDP, and barely get up to 12Mbit/s. I did some timings using UDP and a 1024 byte message using a bind()ed and connect()ed socket: Total time to send message (when sendto() or write() is called until it returns): 650us Time spent in the driver (using atse.c driver, altera_tse.c driver gives similar performance): 140us Time in the driver waiting for the hardware to send data: 80us (seems it's doing 100Mbit instead of GbE, but that's a relatively small issue here considering overall performance) Kernel overhead before entering driver code: ~400us Kernel overhead after exiting driver code: ~100us Timing was done by writing to an output pin and using an osciloscope. It seems there's just a lot of overhead using the linux IP stack on the Nios II. Has anybody been able to get better performance or is it hopeless given the speed of the processor?55 Replies
Altera_Forum--- Quote Start --- Will I never achieve that throughput using UDP, Nios II and the TSE core? --- Quote End --- Supposedly not with a NIOS CPU (there are special CPUs that are designed to be able to do this in router applications such as Ubicom 7K) and especially not with a Linux IP Stack (there are dedicated "Zero-Copy" stacks and special OSes that maybe can do this. Example: Ubicom again) (Beware: Ubicom provides nice cheap chips but the development tools are rather expensive.) -Michael- Altera_Forum
Benjamin,
As stated already, you won't get the speeds you are looking for with Linux. It may be possible but difficult on Nios II. But to try to answer your questions --- Quote Start --- 1. How do I compile the kernel with full optimizations? --- Quote End --- There is a setting for the optimization and debug level to use for the build in menuconfig. --- Quote Start --- 2. How do I increase cache size? --- Quote End --- There are settings for cache size and various other parts of the CPU in SOPC Builder, one of the tools used to generate the FPGA image. --- Quote Start --- 3. How do I decrease memory latency? --- Quote End --- This depends on a lot of things -- speed of the RAM, type of the RAM, speed of the processor, cache size, burst settings, etc. You'll need a lot of back and forth with the hardware/fpga designers on this. Some other sections on this forum may be more helpful on this. If you do choose to go with Linux and manage to get any speed improvements, please post back in this thread. - Altera_Forum
Thank you for the feedback. Just wanted to check that all those settings were in Quartus and not Nios EDS.
Sounds like it's going to be a big challenge to get this working. How do people transfer data quickly to a pc? I find it hard to believe that there are no easy and relatively cheap solutions out there to achieve this. - Altera_Forum
I agree. With Nios II running Linux/uClinux, this will be a big challenge. There is, however, a non-Linux UDP offload example (http://nioswiki.com/index.php?title=exampledesigns/nios2udpoffloadexample) on the Wiki that hits GbE line speed rates.
I feel that this is the best way to do this sort of thing with soft core processors. Regards, --slacker - Altera_Forum
Thanks! I'm going to try figure out how that works and hopefully we can tweak it to our application.
- Altera_Forum
Benjamin, what exactly are you trying to accomplish ?
Three factors are slowing down your transfer rate: Linux, UDP/IP, and NIOS. Do you really need Linux ? There are other options. AFAIK you even can do a TCP/IP stack in Hardware with an FPGA.. If you do need Linux, you might run Linux on one CPU and provide dedicated hardware (perhaps including a CPU) for the communication stack. Do you really need UDP ? Other options include "naked" Ethernet, Firewire and USB3. Those provide a much smaller protocol overhead. Do you really need NIOS. Of course NIOS is a very viable option when you want to include a CPU in an FPGA. Do you really need an FPGA ? Do you really need a "single chip" design ? There are by far faster CPUs done in real Hardware. -Michael - Altera_Forum
Hi Michael,
Thank you for the response. You raise some of the big issues we have been debating over the last few weeks. Our requirement at this stage is quite simple, yet complicated to achieve. We have a custom designed board which acts like a flatbed scanner. It produces 200 MBytes of data in 3 seconds say every ten minutes or so, and we need to transfer that 200 MB in real time to a Mac Mini or equivalent. The 200MB is generated at a constant rate over the three seconds. Our PCB has a Cyclone III on it, connected to a Marvell 88E1111 PHY and Halo GigE Connector. So we have the hardware in place to implement the transfer. At this stage there are no other systems on the network, so it's a point-to-point connection. The system can be set up at installation and then will not change, it does not need to be flexible like a normal scanner. The board will always be connected to the same computer etc. Rather than answer your questions, I might ask what you recommend. We do not need TCP/IP and all the complications and overheads associated with that. UDP made the most sense to us - I'm busy looking into this UDPOffLoadExample (have been sorting out Quartus 9.0 to run it). Do you have any recommendations or advice for us at this stage? I do have some FPGA VHDL programming experience, but it's quite limited and I'm trying to move up the learning curve as quickly as possible. We also do not have unlimited funds at this stage, but purchasing the NIOS and TSE cores at $500 each is possible for us. Thanks! - Altera_Forum
If the hardware can't be modified, the best option supposedly would be not to use Linux and/or not to use IP. There are other IP stacks then the Linux stack. Consulting another Altera Forum and/or an Altera sales FAE might bring up some solutions.
If you don't use IP (much faster) you would need to write a driver for your propriety protocol for the Mac. -Michael - Altera_Forum
Dear All,
I'm facing that kind of problem with a stratixIII development board. It seems to me that I could never expect to do something better than 2 or 3 MB/s still using a 1Gb link and a standard approach. Please,can someone confirm this to me? Thanks a lot in advance. :confused: - Altera_Forum
Is this just a problem with speed, or is it worse? I am seeing actual data corruption on inbound packets using TCP. I can see it running iperf. I have to slow way down to eliminate this. Granted, I am running with only a 1K cache and a non-optimized kernel. I am in the process of fixing these things, but I don't think I should ever see corruption when using TCP...?
Thanks, Van
