lwip MicroC/OS-II throughput

Yes those measurements were on a 100MBps link.

I recently measured LWIP throughput on the standard reference design with the clock frequency raised to 70MHz and the figures are:

UDP TX 820 kbytes/s

UDP Rx 610 kbytes/s

The software use was a very simple sockets application with the board sending frames as quickly as possible to a PC (or visa versa) and averaging how long it takes to send every 200kbytes. The problems here are that the lan91c111 does not have a particularly fast interface (the processor has to copy every word), and the memory used is SDRAM.

It's hard to say if LWIP is suitable for your application without knowing the rates at which you wish to capture the data. The raw LWIP implementation is mainly suitable for use as a control channel. If you're looking to get rates which approach line rate you need to consider some hardware acceleration.

Thanks,

I was hoping that the documented throughput rates were for 10Mbps. This may be a problem.

Hi,

i've finished my LWIP uCOSii UDP test today and i have reached these results:

UDP TX 35.55 Mbit/s (UDP payload rate,packet length 1472 byte)

UDP RX 17.66 Mbit/s (UDP payload rate,packet length 1472 byte)

I've used a Stratix 1S10 development board with a NiosII(f) 100 MHz clock. If you are interested, i can tell you more.

bye

Soin,

I for one would be very interested to know more.

hi rugbybloke

i've used the Stratix 1S10 development with the "full featured" example with some modifications:

-Nios II (f) 100 MHz,Instruction cache:16Kbytes,Data cache:4Kbytes

(also the SDRAM has a 100 MHz clock, with delay...)

-System_clk_timer->Initial period:10 ms (is the tick used by UCOSii)

-debug module level 1

-no onchip ram

Moreover, in the IDE, u have to set "optimization level: -O3" in your program and in the associated library.

My test program uses UNIX socket to only trasmit or receive continuosly (test tx or test rx) UDP packet of variable length: in the tx test i send an incremental counter to a software running in a PC connected through a LAN cross cable with the board, in the rx test i check this counter.

bye

I know that this is a little bit late, but I have done some detailed speed measurement and tuning that may be relevant. The environment is a 50MHz NiosII on a board with a Cyclone 1c20 and a 91c111 - very similar to the 1c20 demo board. Static RAM is used for program and data during execution. A flash is used for boot only. Builds have been done with the 1.1 tools and the Beta 5.0 tools.

The basic system operation is bulk data collection and transfer to a host PC. Data typically arrives in 4K chunks via DMA. The data is quickly reduced to a 2K chunk using a copy operation. The DMA is transferring some useless bits. Headers are then prepended and the data is sent to the PC via TCP. The data streams out with no application level acknowledgement.

Some of the code optimizations have been in for a long time, and I don't have a good baseline measurement without them. I started this work with toolkit 1.0. The optimizations were as follows: memcpy - increase the level of loop unrolling, inet/chksum - unroll the inner loop, 91c111 driver unroll the inner loop of the transmission algorithm. I am considering also unrolling the 91c111 driver's receive inner loop and getting rid of the rx thread entirely. These latter two steps haven't been taken, yet.

I have found that -O3 produces very much the same results as -O2, though it does generate significantly larger code. Space is an issue in this system, so I just use -O2.

Using the 1.1 toolkit, I was able to get about 1.5ms per data chunk, which corresponds to about 11Mbps. With the 5.0 toolkit, I slowed down to about 1.9ms per data chunk, or about 8.5Mbps. Though I bemoan the slowdown with the 5.0 toolkit, I have found that the 1.1.0 lwIP in 5.0 is more robust under packet loss than the 0.7.2 lwIP in the 1.1 toolkit.

The timing measurements were taken with a logic analyzer. I modified os_cpu_c.c to put in some outputs to a port that I could observe. I used one port bit per task, so I was able to get a nice waveform showing active task times. I also added bits to track the time spent in memcpy, the checksum, tcp_write, and tcp_output, though these weren't strictly necessary.

If you really need speed, plan on doing some tuning. Make provisions for measuring time to guide your tuning efforts. If you can get an ethernet controller that operates as a DMA bus master, you should. It's silly to be transcribing data to a fast ethernet chip the way we do with the 91c111.

If you do plan to use UDP rather than TCP, consider just doing it yourself without involving the stack. It's not a big deal. If the data link is one hop over an ethernet, you could also consider dispensing with the UDP checksum. Since you are protected by the ethernet CRC, the UDP checksum adds little.

Avoid transcriptions to the extent possible. Wherever you have a loop processing your bulk data, make sure that it is unrolled.

If you leave time for this in your project, you'll probably enjoy doing it. Speed tuning is kind of fun if you're not under the gun when you're doing it.

Good luck!

This is an update on the reply that I posted a couple of days ago. I've found out why I had such different time measurements between the 1.1 toolkit and the 5.0 toolkit. There was one other modification that I had done to the 1.1 environment that I hadn't bothered to move forward. I had no idea that it had speed implications, but in fact it did.

The 1.1 toolkit's lwIP version had problems in the protection of pbuf's in a multitasking environment. The only way to get effective protection was to specify SYS_LIGHTWEIGHT_PROT. That forced the lwIP library to used interrupt disables rather than a semaphore to protect pbuf's and it's various memory pools. (A couple of other declarations were also required to make this work.)

The 5.0 toolkit's lwIP version resolved the problems in the protection of pbuf's, and the default distribution used semaphores rather than the "lightweight" mechanism. Because of this, I didn't bother to pull forward the changes I had made in lwipopts.h to specify and facilitate SYS_LIGHTWEIGHT_PROT. Yesterday, for reasons unrelated to speed tuning, I had to pull this change into the 5.0 toolkit. Much to my surprise, I got back most of the time that had been lost. It amounted to almost 400us in the handling of two outgoing packets and one incoming ACK. I'm now back in the vicinity of 1.5ms for these operations. This is just under 11Mbps for the payloads in question.

It seems unlikely that semaphores would be so costly, but lwIP does load them down with a mechanism of its own for executing specified functions at specified times. Even with this, though, there must be a lot of semaphore calls in the processing of normal packets.

If you are tuning for performance, I strongly recommend changing to the lightweight protection mechanism. This may well be more important than any of the loop unrolling that I did.

When using LWIP + uCOS, I see there are quite a few tuning options available thru the GUI and in the file opt.h . What changes to these default options are you finding helps the performance? Is there better documentation (other than the code) to explain what some of these do?

I see that some are using a 10ms timer interval for uCos, despite using a fast clock. I believe the HW timer defaults to 1ms, so was there any performance testing done as a reason for the longer interval?

thanks

JimG,

You mention "other declarations" needed to support the lightweight mechanism. What are these?

thanks

I made the following changes which compiled and tested ok, but haven't done any comparative performance measurements yet.

1.) Added the following code to:

\altera\kits\nios2\components\altera_lwip\UCOSII\inc\lwipopts.h

/*

* Enable LightWeight Protection.

* Refer \altera\kits\nios2\components\altera_lwip\UCOSII\src\downloads\lwip-1.1.0\src\include\lwip\sys.h

* for changes to the SYS_ARCH_DECL_PROTECT(), SYS_ARCH_PROTECT() and

* SYS_ARCH_UNPROTECT() macros.

*/# define SYS_LIGHTWEIGHT_PROT 1

2.) Made the following changes in:

\altera\kits\nios2\components\altera_lwip\UCOSII\src\downloads\lwip-1.1.0\src\include\lwip\sys.h

/** SYS_ARCH_DECL_PROTECT

* declare a protection variable. This macro will default to defining a variable of

* type sys_prot_t. If a particular port needs a different implementation, then

* this macro may be defined in sys_arch.h.

*/

//#define SYS_ARCH_DECL_PROTECT(lev) sys_prot_t lev# define SYS_ARCH_DECL_PROTECT(lev) alt_irq_context lev

/** SYS_ARCH_PROTECT

* Perform a "fast" protect. This could be implemented by

* disabling interrupts for an embedded system or by using a semaphore or

* mutex. The implementation should allow calling SYS_ARCH_PROTECT when

* already protected. The old protection level is returned in the variable

* "lev". This macro will default to calling the sys_arch_protect() function

* which should be implemented in sys_arch.c. If a particular port needs a

* different implementation, then this macro may be defined in sys_arch.h

*/

//#define SYS_ARCH_PROTECT(lev) lev = sys_arch_protect()# define SYS_ARCH_PROTECT(lev) lev = alt_irq_disable_all()

/** SYS_ARCH_UNPROTECT

* Perform a "fast" set of the protection level to "lev". This could be

* implemented by setting the interrupt level to "lev" within the MACRO or by

* using a semaphore or mutex. This macro will default to calling the

* sys_arch_unprotect() function which should be implemented in

* sys_arch.c. If a particular port needs a different implementation, then

* this macro may be defined in sys_arch.h

*/

//#define SYS_ARCH_UNPROTECT(lev) sys_arch_unprotect(lev)# define SYS_ARCH_UNPROTECT(lev) alt_irq_enable_all(lev)

//sys_prot_t sys_arch_protect(void);

//void sys_arch_unprotect(sys_prot_t pval);

3.) Added the following include to:

\altera\kits\nios2\components\altera_lwip\UCOSII\src\downloads\lwip-1.1.0\src\core\pbuf.c

\altera\kits\nios2\components\altera_lwip\UCOSII\src\downloads\lwip-1.1.0\src\core\memp.c

# include "sys/alt_irq.h"

Not sure about any of the other tuning options, haven't got that far yet.

ciao