Forum Discussion

Altera_Forum's avatar
Altera_Forum
Icon for Honored Contributor rankHonored Contributor
14 years ago

Network speed between to 2 processors

I have an interesting issue that i can't get to the bottom of. So i have written software to transmit data at a consistent 32Mbps to a Windows PC this works well. Occasionally i see some delay in sending the data (seen through the FIFO overflowing on on the contoller board) which i suspect is Windows doing its own thing.

To see if this is a Windows related problem as i suspect i modified the software to act as a pure reciever. I created a quick test app in Windows with virtually identical data size and tested. This runs at a healthy 50Mbps and so should give no problems keeping up with the other data being sent by the other controller board.

Now heres the oddity, if i connect the two controller boards together my Tx Fifo is always overfilling!!! I have no idea why at the minute, they have different IP's and MAC Addresses and run through the same set of switches as the PC to FPGA setups.

Has anyone seen something similar to this? Or have any ideas on what could be causing this? The throughput of both boards is good enough when connected to a PC but not when connected to each other.

Cheers

Tim

10 Replies

  • Altera_Forum's avatar
    Altera_Forum
    Icon for Honored Contributor rankHonored Contributor
    14 years ago

    Is it TCP or UDP? Are you using a software stack or directly sending data from hardware?

  • Altera_Forum's avatar
    Altera_Forum
    Icon for Honored Contributor rankHonored Contributor
    14 years ago

    The connection is TCP, and both units run the MicroC os and iNiche stack. The design is based off of the Cyclone 3 dev kit. This really doesnt make sense to me that both boards in isolation work well but when used together fail. (Well not fail but perform poorly).

    Any ideas would be great.

    Cheers

    Tim
  • Altera_Forum's avatar
    Altera_Forum
    Icon for Honored Contributor rankHonored Contributor
    14 years ago

    Try to put a PC with an Ethernet spy application such as Wireshark to see what kind of packets run between the two boards. Look out especially for TCP window size adjustments packets. I had the problem once, with the TSE's rx buffer filling too fast and some packets were lost. If you can't reduce the TCP window size in the stack (I don't know if it's possible) increasing the rx buffer size to something above the window size can improve the traffic a lot in some situations.

  • Altera_Forum's avatar
    Altera_Forum
    Icon for Honored Contributor rankHonored Contributor
    14 years ago

    Hi Daixiwen,

    I'll try and bridge the connection across my network cards and see if i can put a board on each side to see what goes on. The strange thing as i said is that the two boards seem to be able to Tx and Rx with Windows well. Its only when connected together i get the problem. The through put for Tx is a fixed 32Mbps and the Rx is capable of nearly 90Mbps flat out.

    I'll bridge the adapters and see if wireshark has any leads.

    Thanks

    Tim
  • Altera_Forum's avatar
    Altera_Forum
    Icon for Honored Contributor rankHonored Contributor
    14 years ago

    when I had the TCP window size problem the throughput depended a lot an what the board was connected to. Even changing from Windows to Linux on the PC changed the results. But even if it isn't a window problem, having a look at the packets could give indications about what's wrong.

  • Altera_Forum's avatar
    Altera_Forum
    Icon for Honored Contributor rankHonored Contributor
    14 years ago

    The plot thickens!!!

    I set up my PC to act as bridge so i could use wireshark to monitor the data between the controller boards. In this configuration it appears to work as well. Meaning that only when connected together do they have a problem.

    I had a look at the window size and couldn't see anything alarming and all checksums and acks appear to be there. Is there anything particular i need to look for. TCP is not my area of expertise.

    Maybe the PC's adapters are correcting something? But i would have hoped wireshark might have flagged it.

    Cheers

    Tim
  • Altera_Forum's avatar
    Altera_Forum
    Icon for Honored Contributor rankHonored Contributor
    14 years ago

    If you can use a hub or a switch with a promiscuous port, try to use that instead. It is indeed possible that the PC acting as a bridge changes the TCP streams in some way. It depends on how the bridging is done.

  • Altera_Forum's avatar
    Altera_Forum
    Icon for Honored Contributor rankHonored Contributor
    14 years ago

    With correct software the transmit fifo shouldn't be overfilled - since it is an entirely local matter. So are you seeing an error due to it being overfilled, or just getting a transmit requests deferred due to transient lack of space?

    Transient lack of space might just be caused by trying to send a lot of packets at once (as might happen with a very large transmit window), especially if the MAC unit isn't being told to actually transmit the packets immediately (it can't be told until the dma? copy of the data into the fifo is complete [1]).

    It might be worth doing some tests with a slow network - where is is easy to exceed the network speed - to check the driver (etc) behaves properly.

    If you are connecting the systems with a cross over cable, check that the PHYs have actually negotiated the expected speed.

    [1] Latency can be improved by giving an early indication - but that can lead to transmit underruns if the dma is stalled for any reason (eg bus contention).
  • Altera_Forum's avatar
    Altera_Forum
    Icon for Honored Contributor rankHonored Contributor
    14 years ago

    Hi dsl,

    Thanks for you reply. I should probably clarify the FIFO buffer is not part of the Network stack it is an internal FIFO for queueing up some samples data from ADC's. This has a constant rate of 32Mbps. I have the tested the controller board can constantly send at around 90Mbps (dummy data) for several days.

    The network doesnt appear to lose any data or give any errors to indicate a problem it just runs too slow when connected together. I will try and get a Hub from somewhere to see if we have any errors. I have checked and they both negotiate correctly, when connected together (although i have always used Switches between controllers.

    Cheers

    Tim
  • Altera_Forum's avatar
    Altera_Forum
    Icon for Honored Contributor rankHonored Contributor
    14 years ago

    It might then be worth looking at whether the TCP stack is actually ever doing any retries - and then try to track down where the packets get dropped. If you can't buffer enough data to allow for a retry it might be better to use UDP - and possibly some simple, fast, retransmit algorythm.

    Guessing, it might be that the receiver discards packets when they are very close together - and the PC bridge manages to separate them.