TSE or SGDMA related problem with concurrent tx and rx

More information... and more help request

I think it is definitely a tse device issue. Infact this is what I discovered:

I directly inspected the tse statistic counters (tse base address + 0x60 to 0xe0) after sending 14 packets which were looped back by my external test device.

aFramesTransmittedOK = 14

aFramesReceivedOK = 9

aFrameCheckSequenceErrors = 0

aAlignmentErrors = 0

ifInErrors = 5

ifOutErrors = 0

etherStatsJabbers = 0

etherStatsFragments = 5

This data matches with my application result: only 9 packets received out of 14.

The etherStatsFragments counter tells me the 5 lost packets were too short frames with crc error.

How can this happen? I'm not an ethernet guru and I don't know the mac and phy internal processes.

Could it be that the part of rx frame data which overlaps with a tx frame will eventually get lost? As if the device was working in a half duplex mode (which is not the case)

How can I overcome the problem? Is there any special configuration in the tse I'm missing?

Thank you for any help

That's strange... could it be noise going from the tx line to the rx line?

You could put some signaltap probes on the MII interface (between the MAC and the PHY) to see exactly what is sent and receive, and compare them. That way you'll know if the problem is inside the TSE MAC or further out.

Thank you for the answer

I had just made what you suggest and I discovered it's probably related to noise. IMHO is not related to fpga but to ethernet phy.

I'm using a DBC3C40 dev board which have a National DP83640 phy.

I monitored the CRS signal and I see that when I get the rx error CRS goes low before the complete frame has been received. When tx and rx frames don't overlap CRS behaves correctly, as far as I could test.

This is the timing:

time 0: start transmission of tx_frame

time 640ns: start receiving rx_frame (crs goes high)

time 5.36us: end of tx_frame

time 6.32us: end of tx_frame

When the frame is correctly received from my application, crs goes low at time 6.32us. When I get the error, crs goes low at time=2.40us (this is quite deterministic).

I changed the ethernet cable but the behaviour is always the same. On the dev board there's a second DBC3C40 device available and I could try to switch the design on this one. Anyway I'd like to understand why it fails

Found the problem cause!!!

But now I really need help from you expert guys.

Actually there's a bug in the RMII2MII block supplied with the dev board; I attached the vhd file.

The block defines two internal signals rx_clk and tx_clk but there is some sort of synchronization issue with rx_clk: the block restarts it when an input frame is received (triggered by crs high) but this clk is also used as output clock for the tse mac.

Depending on frame arrival time (crs on) the mac_rx_tx_clk will go half period out of phase (then the frame error) or remain in phase with tx_clk (no error case)

I don't know much of MII interface; can anyone suggest me how can I modify the vhd code to fix the bug?

Or better: Are there any other RMII2MII converters available?

Thank you

Cris

The other problem with that code is that it is generating a gated clock for the MAC. It isn't recommended and can lead to problems. I would rewrite the converter to use two clocks generated from a PLL instead. Unfortunately I don't have any card with a RMII PHY so I can't help you.

There seems to be a Verilog code posted here (that obviously needs some serious reformatting): http://www.opencores.org/forum,ethernet%20mac,0,1407. I don't know if it works, and I find it rather strange to use two clocks in the sensitivity list. I'm not sure it is synthesizable.

--- Quote Start ---

The other problem with that code is that it is generating a gated clock for the MAC. It isn't recommended and can lead to problems. I would rewrite the converter to use two clocks generated from a PLL instead.

--- Quote End ---

Hi Daixiwen,

I replaced the original gated clock for the MAC with another one which is simply generated from the input clock dividing by 2 without a pll. Now it works perfectly, although I've not tested it for a long time, yet.

Probably this solution is not that good and it may be susceptible to timing problems upon reprocessing.

In case of further problems I'll try to use the pll solution as you suggest.

Thank you

I return on this old post because I discovered the above problem is still present in some situations.

I hope Daixiwen or other Altera Gurus have any hints for me.

I think the problem is not with mac or phy, but with dma.

Infact I switched from TSE to opencores mac and I have exactly the same behaviour.

I can reproduce in a deterministic way a condition where the tx frame gets corrupted if a rx frame is received in the same time.

Here is more exaclty what I see:

I monitor txen and txd signals that drive the phy, and crs or mac_rxdv for identifying rx frame timing.

Tx frame is about 400bytes long (txen pulse about 32us).

Rx frame starts being received a few us after txen rising edge.

On most cases txen length is correct; same for data on txd. While in many cases I see corrupted data on txd, as if it has been overwritten with rx data. In consequence of this, also the txen pulse becomes lenghtened or shortened.

As I said in previous posts this only happens if I have rx data.

I checked the mac txclk, too, but it's perfectly in phase and it doesn't miss any cycle.

I remarked that the first 240-270 bytes of the tx frame are ALWAYS correct: the data corruption always happen from a random position AFTER this threshold.

I don't know the dma descriptors inner workings; probably 256bytes is the size of a single descriptor? So I suppose I have some problem with descriptor chaining.

Please help.

Thank you in advance for any support

The SGDMA buffers in the descriptors can be up to 64k long. That said I faced a similar problem when improving/debugging the TSE drivers for eCOS. I had a bug in buffer allocation and some times a received packet would end up in a buffer used for transmit.

Are you using the standard altera TSE drivers with uCOS/Interniche? In that case, what version?

You could add some debug messages displaying the buffer address and length used during each DMA transfer, just to check that you don't have any overlap.

It could also be a cache coherency problem. Check that the driver properly flushes the cache before a DMA read and invalidates it after the DMA write (or uses the alt_remap_uncached() functions).

Hi Daixiwen, thanks for answering me.

As I said before, I had the same problem both with TSE and with OpenCores MAC, which is what I currently use in this design.

Apart the mac, I use the standard uCOS/iniche.

I have two ethernet ports in my design.

The only diffence from standard usage is that on one port I send raw Ethernet packet eth the net->pkt_send function.

On other port I have no problems, but in this case I think the TCP protocol automatically fixes it.

Anyway, yesterday I found a way to solve the problem.

I defined ETH_OCM_SYNC_TX in OpenCores mac driver, so that drive syncronously transmits and blocks until frame is transmitted, rather than using interrupts.

It now uses the net->raw_send function rather then net->pkt_send.

Now everything is working perfectly; so it was really a driver problem when managing concurrent tx and rx data.

The performance has not decreased. I only have minimal performance drawbacks on the other ethernet port (the one using the tcp/ip stack), so I could use this as the final solution. I think I can also modify the driver in order to use sync mode only on the first port.

Regarding your message:

Version: I use Nios IDE 9.0sp2 with ucos and iniche which come with it.

Cache: I assumed the standard driver already flushed cache; I've not driver knowledge, so I don't know how to check if flushing is performed, nor how to force it.

Debug: same as above, I really don't know where I need to place the messages; maybe where putq and getq are called?

Cris

For the debug messages; I was more thinking about the tse_mac_raw_send() and tse_mac_rcv() functions in the driver.

I assumed too that the Altera drivers properly handle the data cache, I was just wondering if you were using a more "exotic" OS.

I just had a look at the driver, and from what I see, it requires uncached buffers. The Interniche stack is set up properly to use them, but if you directly call the raw_send() function, you need to use an uncached buffer. If you don't you'll run into cache problems one day. I haven't looked at the opencores driver but I guess it is the same thing.