Forum Discussion
Altera_Forum
Honored Contributor
Honored ContributorWeird Ethernet problem
Premise: I don't know if this weird problem is related to TSE, sgdma or Nios code. Please help me to identify, or better, to solve it.
I'm using Nios II with tse mac and National Semi DP83640 phy. I trasmit and receive Ethernet frames and sometimes tx or rx frames are lost and I can't understand why. I monitored both the Ethernet link and the TSE-phy signals and I'm testing with a system that immediately loops back the rx frame. This is what I see: - the txen signal from the mac is always correctly generated - the rxen signal (same behavior for crs signal, carrier sense) sometimes is lost - often, when rxen is missing, I can neither see any frame transmitted on the cable, although sometimes I do. And now the weirdest part: The error seems to be almost deterministic if a rx frame arrives just after I have transmitted a frame, say within a few microsec, while it never happens if the rx frame is superimposed to tx or even if it is delayed. So, if I use a direct loopback I seldom have any error, while the error is systematic if I insert an Ethernet hub, which delays tx/rx frames about 20us I simplified my tx function in the following way, in order to debug with a single constant data frame, but nothing has changed.
void test_frame(void)
{
PACKET outpkt;
int len;
len = 40;
LOCK_NET_RESOURCE(FREEQ_RESID);
outpkt = pk_alloc(len+20);
UNLOCK_NET_RESOURCE(FREEQ_RESID);
if (!outpkt) {
printf("Error: pk_alloc() failed");
dtrap();
return;
}
memset(outpkt->nb_buff, 0, len+ETHHDR_SIZE+ETHHDR_BIAS);
/*
here I init packet with my constant data
*/
outpkt->nb_plen = ETHHDR_BIAS + len;
if (outpkt->net->pkt_send) {
outpkt->nb_prot = outpkt->nb_buff;
outpkt->net->pkt_send(outpkt);
}
else {
outpkt->net->raw_send(ecat_net, outpkt->nb_buff, outpkt->nb_plen);
LOCK_NET_RESOURCE(FREEQ_RESID);
pk_free(outpkt);
UNLOCK_NET_RESOURCE(FREEQ_RESID);
}
}
11 Replies
Altera_ForumCris, I'm also using TSE but with Marvell 88E1111. I had sometimes losts in transmitting frames. I used Signaltap to identify the problem. The reason in my case was the transmit FIFO size. It was to small. I chaged the size from 1k to 4 kByte.
I checked the afull signal of the Transmit FIFO (tse_mac:the_tse_mac|altera_tse_mac:altera_tse_mac_inst|altera_tse_top_gen_host:top_gen_host_inst|altera_tse_top_w_fifo_10_100_1000:U_MAC_TOP|altera_tse_top_w_fifo:U_MAC|tx_a_full) The backpreassure feature of the streaming interface from SGDMA to the TSE_MAC was obviously not working so the SGDMA hang-up. May be you look at the control signals of tx and rx FIFOs with trigger on rxen in Signaltap. Jens- Altera_Forum
Is the link set up in half duplex mode when you are using a hub? In that case could it simply be that you have collisions?
- Altera_Forum--- Quote Start --- Is the link set up in half duplex mode when you are using a hub? In that case could it simply be that you have collisions? --- Quote End --- No, the link is always in full duplex mode. As I said, the weirdest part is that the problem arises when rx and tx are not at the same time; indeed I don't see this anomaly if I receive the frame exactly when I'm transmitting another one (i.e. when rx_en and tx_en tse mac signals are both active). I can now reproduce in a deterministic way these conditions: - rx and tx contemporary: OK - rx delayed about 15us from tx: OK if tx rate is more than 1 every 50ms. If I use a higher tx rate, some frames are lost. In this last condition, if I transmit a frame every 5ms and the same frame loops back after 15us I obtain this cyclic behaviour: frame are transmitted correctly for about 1s then stop for about 0.5s, then again it works for 1s and so on. At the mac-phy level I don't see any anomaly, although I did't analyse actual data, but only checked the presence of tx_en and rx_en signals. From this last result I think that there could be a problem with mac clock timing. This is a sort of frequency beating behavior. Regards
- Altera_Forum
I don't think that the problem comes from the Mac, the rx_en signal being generated by the PHY... How does it happen exactly? Is the rx_en signal suddenly de-asserted in the middle of a packet, or is it entirely de-asserted?
Here are a few possible causes I could think of: - bad connection between the phy and the FPGA, or noise/coupling on some signals - the MAC and PHY are'nt using the same clock to communicate on the MII bus - bad configuration of the PHY (MDIO registers) - power supply problems on the PHY (bad filtering and/or decoupling) - Altera_Forum--- Quote Start --- How does it happen exactly? Is the rx_en signal suddenly de-asserted in the middle of a packet, or is it entirely de-asserted? --- Quote End --- When there's the problem rx_en is always low, it doesn't become de-asserted in the middle of packet. I also checked the rxerr signal from phy but it signals nothing. --- Quote Start --- Here are a few possible causes I could think of: - bad connection between the phy and the FPGA, or noise/coupling on some signals --- Quote End --- I use a off-the-shelf development board (DBC3C40), so I exclude connection problems. Moreover the board has two ethernet ports and both show exactly the same behaviour. --- Quote Start --- - the MAC and PHY are'nt using the same clock to communicate on the MII bus --- Quote End --- I tried several clock design: same clock for mac and phy, pll for clock regeneration, phase shifting them and so on, but so far I again obtained always the same behaviour. --- Quote Start --- - bad configuration of the PHY (MDIO registers) --- Quote End --- I used the standard configuration provided with the driver. Can you suggest me anything I can change? I read the phy datasheet but I couldn't find anything that can be related to my problem. --- Quote Start --- - power supply problems on the PHY (bad filtering and/or decoupling) --- Quote End --- I'd exclude for the former reason. Should I trust the board supplier or not? Regards.
- Altera_Forum--- Quote Start --- I can now reproduce in a deterministic way these conditions: - rx and tx contemporary: OK - rx delayed about 15us from tx: OK if tx rate is more than 1 every 50ms. If I use a higher tx rate, some frames are lost. In this last condition, if I transmit a frame every 5ms and the same frame loops back after 15us I obtain this cyclic behaviour: frame are transmitted correctly for about 1s then stop for about 0.5s, then again it works for 1s and so on. At the mac-phy level I don't see any anomaly, although I did't analyse actual data, but only checked the presence of tx_en and rx_en signals. From this last result I think that there could be a problem with mac clock timing. This is a sort of frequency beating behavior. --- Quote End --- Sounds like a data cache problem. Be sure to use non-cached addresses with the TSE. Bill
- Altera_Forum--- Quote Start --- I used the standard configuration provided with the driver. --- Quote End --- Which/whose driver? Bill
- Altera_Forum--- Quote Start --- Sounds like a data cache problem. Be sure to use non-cached addresses with the TSE. --- Quote End --- Thank you for your answer Bill. Where do I find this cache configuration? I use a standard design provided with my dev board, with minor changes to fit my needs: - Nios II/f core - TSE MAC + sgdma rx/tx + on chip memory descriptor memory - Micrium RTOS II + Interniche stack The only reference to cache I could find is in Nios configuration in sopc builder. Should I disable it? Regarding your question about the driver, all mac and phy initialization is performed by OS and stack: that's why is said it was the standard one. Cris
- Altera_Forum--- Quote Start --- Thank you for your answer Bill. Where do I find this cache configuration? --- Quote End --- I meant to be sure to use non-cached addresses with SGDMA. With the Altera TSE and Interniche you don't need to worry about this. --- Quote Start --- I use a standard design provided with my dev board, with minor changes to fit my needs: - Nios II/f core - TSE MAC + sgdma rx/tx + on chip memory descriptor memory - Micrium RTOS II + Interniche stack --- Quote End --- This could be different in NIOS II 9.x but in 8.x the National Semi DP83640 is not listed in the supported PHY table. Do you have an alt_tse_phy_profile entry for it, or can you confirm the entry being used is identical to the DP83640? Just because this came to you as a kit and/or with drivers doesn't mean it's correct. Our development board (I think from the same company as yours) had a hardware design flaw, and the supplied TSE and PHY Altera drivers are NOT bug free. --- Quote Start --- The only reference to cache I could find is in Nios configuration in sopc builder. Should I disable it? --- Quote End --- No. Performance will be shot doing so. --- Quote Start --- Regarding your question about the driver, all mac and phy initialization is performed by OS and stack: that's why is said it was the standard one. --- Quote End --- Got it. Don't take that to mean it's bug-free. You could be doing something that is exposing a bug. Bill
- Altera_Forum
Thank you for your answers Bill.
I definitely think it's a driver or cache problem as you suggested. If you have any other advice I'll be grateful, since I've been stuck on this point for a few days so far. This is my latest results: I'd exclude hw/fpga problems: I reduced Nios frequency,removed data cache,changed Quartus project fitting effort, timing constraints and clock distribution system but I always obtained exactly the same behaviour. I kept on testing with the problematic case: rx frame arriving 15us after sending the tx frame (both 60 bytes long, about 6us required for transmission) - tx frame period = 5ms: frames correctly transmitted for 510ms, then it doesn't work for 330ms, then again ok for 510ms and so on - tx frame period = 10ms: similar behaviour but ok/bad times are splitted 650/190 - tx frame period >= 25ms: ok Note that the period is always 840ms: why? Can this suggest the cause of the problem? Finally: remind that if tx and rx frames are almost overlapped on wire (<5us skew) or are very separated (>50us) the system works perfectly, whatever the frame rate is!!! Regarding the phy driver, I use Quartus 9.0sp2 and this actually supports only National DP83865 and DP83848C. My dev board came with a patch of altera_avalon_tse.c in order to support the DP83640 phy. I'll check this file for bugs Regards Cris
