Ethernet roundtrip time with NicheStack & Nios-II

tens of ms seems a lot. A NicheStack/Nios II system isn't very fast, but it should be more than capable of sending more than 10 packets a second. What Ethernet hardware are you using? The driver could use a polling mode instead of interrupt, which could explain the long time before it gets a packet.

You could also use a profiler to see where the CPU is spending some time.

Other things to look into that can affect speed are the RAM interface, CPU cache (or lack of) and optimizations used during compiling (going from -O0 to -O2 has a significant impact).

I run NicheStack with MicroC OS-II on a 100MHz Nios-II/f with cache on a Cyclone III FPGA and I get this figures:

- UDP (ping) : less than 1ms between request and reply, measured on the wire

- TCP : less than 5ms between request and reply, mainly due to application; actually the delay due to TCP Stack and OS is limited to about 2ms

As Daixiwen pointed out, you get major improvements with -O2 build optimization, fast Nios core with cache and higher cpu frequency.

Anyway, even with a basic system like yours (50Mhz Nios, posibly no cache, debug build configuration without optimization) IIRC you should achieve a UDP round trip time far below 50ms.

With UDP do you mean a simple "ping" command or are you implementing a UDP server in your application?

Thanks Cris72 and Daixiwen.

The UDP setup: PC with a RTOS is a very simple UDP client which sends UDP packets. 128 bytes payload. Nios-II app is a modified UDP server based on "Simple Socket Server" TCP template. Current round-trip time as measured on the PC client side (using RTOS APIs) is about 30-40 ms. Board is "Cyclone IV GX Transceiver Started Kit" with original SOPC.

SOPC screenshot is attached. One of the tutorials mentioned using "sdram" for "linker section names". I noticed that the board does not have "sdram"; it has "onchip_ram" and "ssram", to which the CPU intr/data masters are connected. Could it be the mem speed issue due to this arrangement?

Will set -O2 optimization. Will also increase CPU clk and look into CPU profiling.

Any idea how to check if the TSE IP driver is interrupt-driven or is polling?

We are trying to get round-trip time in sub-millisecond range.

-swguy

If you are using the TSE driver with ucOS and the Interniche TCP/IP stack (the default with the Nios IDS) then it is using interrupts.

Are you measuring the delay for several UDP packets or just the first one? The round-trip time for the first UDP packet may be longer because you also need an ARP request and reply. You could also use a sniffer such as Wireshark to see what is happening on the Ethernet level, but I think profiling should give you also a lot of information.

Sending 10,000 packets for the test. First 500 packets are not used in the stats. We are using another RTOS on PC and the RTOS uses an Ethernet card which the Windows does not "see". Wireshark media specific capturing does not list the RTOS we use, but will try.

Incidentally, the Ref. design binaries for "Accelerating Ethernet Performance", when run on Stratix-II Nios-II development board shows roundtrip time of under a millisecond. The design does have number of optimization, including checksum in hardware. The document is old as the board is not supported anymore, but pointers do help nevertheless.

thanks,

-swguy

If you have a manageable switch somewhere and can configure one of its port as a mirror, then you can plug it between your PC and the FPGA board, and monitor that port with a second PC and Wireshark. It can seem overkill but Ethernet sniffers are great debugging tools.

Checksums on 128 bytes payload shouldn't take that long, and besides it is optional with UDP. If possible, you could try and see if you can configure the RTOS on the PC so that it doesn't generate the checksum and check if it changes anything on the round trip time.

I could bring down the round-trip time significantly ~ 60-70% by doing few things specified in AN440 and from pointers by Daixiwen and Chris72. Thanks folks.

The round-trip time is still in few milliseconds. The Cyclone IV device and the eval board has limited resources. So I could not implement "Fast packet memory" and "Checksum in hardware". Surprisingly with this board, anything higher than 50 Mhz did not work either.

Now I have moved to Nios-II eval board, Stratix-II edition. I think Altera no longer supports it, but I could find the source files for AN440 targeting this board. Managed to generate SOF and ELF. Download worked as well. Ethernet link gets established and it is 100 mbps, full-duplex. No errors. However, Nios-II console displays IP addr of 0.0.0.0 during startup.

TCP/IP configuration is correct on the host system and I am using crossover cable for point-to-point ethernet. DHCP is off in BSP.

Any idea why would the IP addr get stuck at 0.0.0.0?

thanks,

-swguy

BTW, I am using MoreThanIP Marvell 10/100/1000 Ethernet daughter card on PROTO2 expansion connector of the board.

-swguy

A Cyclone IV is definitely able to make a Nios II system with Ethernet run at 100MHz. Did you specify correct timing requirements for Timequest? You can usr the timing advisor to check your project for configuration changes to optimize timing.

I also noticed that adding an Avalon MM Pipeline before the Nios CPU JTAG debug interface helps timing.

As for the 0.0.0.0 IP address, if you disabled the DHCP then you need to specify a static IP address. It is done in one of the source files, in a function called something like get_my_ip_address(). I don't remember the exact name, I haven't used the Interniche stack in a while.

A simple custom instruction (add the two 16bit halves of 'a' onto 'b') will significantly improve the checksum time (because the nios has no carry flag so can't do 32bit 'add with carry usually used for sw checksum).

You probably also want one for byteswap (16bit and 32bit).