Hi all, I am trying to test board-to-board communication using a complete 10G Ethernet design that includes an Altera 10 GbE MAC + an Altera 10GBASE-R IP cores. My devices are Stratix IV GT and my boards are two identical Transceiver Signal Integrity Development Kits (EP4S100G2F40I1). I found a hardware demonstration design from Altera Wiki: http://www.alterawiki.com/wiki/10g_ethernet_and_10g_base_r_phy_interoperability_hardware_demonstration_design This design is for different board (100G SIV GT Development Board). So, I have changed pin assignment, regenerate as well as recompile the design for my board. Finally, I could run the demo script to test several loopback modes. My next plan is to test board-to-board communication with the two boards rather than looping back on the same board using the provided TCL scripts. As I am new to such task and there are a lot to grasp, I wonder where I should start to realize my purpose. So, if anyone has experience with this or similar designs, please give me a clue or some advice. Any help will be really appreciated. Thanks for reading this thread.

--- Quote Start --- My next plan is to test board-to-board communication with the two boards rather than looping back on the same board using the provided TCL scripts. As I am new to such task and there are a lot to grasp, I wonder where I should start to realize my purpose. --- Quote End --- You can probably run exactly the same design on both boards, and just connect them together. Try it and see what happens. If nothing works, then read the Tcl scripts and see what they are doing. Your current question is a little vague. Most people will not have your hardware setup, so cannot duplicate your system. So just start using the hardware and see how far you get. If you cannot understand what a particular SystemConsole Tcl sequence is doing, then you can ask specific questions, and you'll be more likely to get answers. Cheers, Dave

Thanks for your suggestions. Actually, I decide to use this example design because it provides not only the complete 10Gbps Ethernet subsystem but also the traffic controller and system controller so that I can simulate as well as test in hardware. And if I can use this for my boards and set up a board-to-board communication, I would only have to modify the MAC for my target application. --- Quote Start --- You can probably run exactly the same design on both boards, and just connect them together. Try it and see what happens. If nothing works, then read the Tcl scripts and see what they are doing. --- Quote End --- I am trying to make some minor changes in Tcl scripts to disable the Loopback modes, and then try to test with local loopback (things happen on only one board with SMA connectors) first. If this works, I will then learn how to write a script for testing board-to-board mode. --- Quote Start --- Your current question is a little vague. Most people will not have your hardware setup, so cannot duplicate your system. --- Quote End --- It seems like no one uses same setup as me, unfortunately. --- Quote Start --- If you cannot understand what a particular SystemConsole Tcl sequence is doing, then you can ask specific questions, and you'll be more likely to get answers. --- Quote End --- I am learning Tcl scripting. Thank you.

I modified the scripts and made it work with local loopback (using SMA cables in one board). I then created a script to work with board-to-board mode and only the transmitter works that gives statistics in System Console. But the receiver is not working. I cannot write to register and the error msg is as below: error: This transaction did not complete in 60 seconds. System Console is giving up. while executing "master_write_32 {/devices/EP4S(100G2|40G2)@1#USB-0/(link)/JTAG/(110:132 v1# 0)/phy_0/master} {0x1000004} {0x00000001}" while executing "master_write_32 $port_id $address $wdata" If you have any idea about the reason for this? Thanks and wait for your suggestions.

--- Quote Start --- I cannot write to register --- Quote End --- If you have two boards connected to one PC, then you need to make sure SystemConsole only uses one board at a time. Look at the Tcl scripts and see how the scripts track whether the 'master' service is open. With two boards, you need to open the 'master' service on the board, perform the read or write transaction, and then close the service. That way you can then read or write to the 'master' service on the other board. Cheers, Dave

--- Quote Start --- error: This transaction did not complete in 60 seconds. System Console is giving up. while executing "master_write_32 {/devices/EP4S(100G2|40G2)@1#USB-0/(link)/JTAG/(110:132 v1# 0)/phy_0/master} {0x1000004} {0x00000001}" while executing "master_write_32 $port_id $address $wdata" --- Quote End --- This error message indicates that SystemConsole did not get a response from the JTAG avalon master you are using. First please rule out all the common failures - there should be no timing analysis failures from Quartus; the clock should be running and reset should not be held asserted. Can you write to any other slaves using this master? If so then the problem may be in the slave which you have at address 0x1000000 so that might be worth investigating.

10G Ethernet and 10G Base-R PHY Hardware Demonstration Design

18 Replies

Altera_Forum

Honored Contributor

14 years ago

From your code:


ETH10G_TOP  ETH10G_TOP (
    .mm_clk_clk                            (clk_50),
    .mm_reset_reset_n                    (RESET_N),
                                        
    .mm_pipeline_bridge_address            (slave_if_address),
    .mm_pipeline_bridge_waitrequest        (slave_if_waitrequest),
    .mm_pipeline_bridge_read            (slave_if_read),
    .mm_pipeline_bridge_readdata        (slave_if_readdata),
    .mm_pipeline_bridge_write            (slave_if_write),
    .mm_pipeline_bridge_writedata        (slave_if_writedata),
    
    .xgmii_rx_clk_clk                    (rx_phy_clk),
    .link_fault_status_xgmii_rx_data    (link_fault_status),
    
    .tx_serial_data_export                (TXP_10G_BASE_R),
    .rx_serial_data_export                (RXP_10G_BASE_R),
    
    .ref_clk_clk                        (REFCLK),
    .ref_reset_reset_n                    (RESET_N),
    
    .avalon_st_rxstatus_valid            (rx_status_valid),
    .avalon_st_rxstatus_data            (rx_status_data),
    .avalon_st_rxstatus_error            (rx_status_error),
    .avalon_st_txstatus_valid            (),
    .avalon_st_txstatus_data            (),
    .avalon_st_txstatus_error            (),
    
    .tx_ready_export                    (tx_pma_ready),
    .rx_ready_export                    (rx_pma_ready),
    
    .rx_sc_fifo_out_data                (avalon_st_rx_data),
    .rx_sc_fifo_out_valid                (avalon_st_rx_valid),
    .rx_sc_fifo_out_ready                (avalon_st_rx_ready),
    .rx_sc_fifo_out_startofpacket        (avalon_st_rx_sop),
    .rx_sc_fifo_out_endofpacket            (avalon_st_rx_eop),
    .rx_sc_fifo_out_empty                (avalon_st_rx_empty),
    .rx_sc_fifo_out_error                (avalon_st_rx_error),
    
    .tx_sc_fifo_in_data                    (avalon_st_tx_data),
    .tx_sc_fifo_in_valid                (avalon_st_tx_valid),
    .tx_sc_fifo_in_ready                (avalon_st_tx_ready),
    .tx_sc_fifo_in_startofpacket        (avalon_st_tx_sop),
    .tx_sc_fifo_in_endofpacket            (avalon_st_tx_eop),
    .tx_sc_fifo_in_empty                (avalon_st_tx_empty),
    .tx_sc_fifo_in_error                (avalon_st_tx_error),
    
    .mdio_mdc                           (mdc_from_the_mdio),
    .mdio_mdio_in                        (mdio_in_to_the_mdio),
    .mdio_mdio_out                        (mdio_out_from_the_mdio),
    .mdio_mdio_oen                       (mdio_oen_from_the_mdio),    
    
    .tx_clk_clk                            (rx_phy_clk),
    .tx_reset_reset_n                    (RESET_N) 
);

There is both an MDIO interface and an Avalon slave interface. The slave should show up in your address map, and there should be registers for an MDIO controller (I assume this is an I2C controller that never actually exits the FPGA).

Cheers,

Dave

Altera_Forum
Honored Contributor
14 years ago
You can check the Qsys system in the attachment. They use a Merlin Master Translator and there is only one Master-Slave connection here is between Merlin Master Translater->Eth_10g mm_pipeline_bridge (slave). I don't know why they do not use the Jtag to Avalon as a Master? What is the role of the Merlin Master Translator? is it used here like a Jtag to Avalon Master?
ETH10G_TOP.zip2 KB
Altera_Forum
Honored Contributor
14 years ago
In the scripts for hardware test, there are many base addresses for accessing. Many of those are not Qsys components.

-- System Controller Base Address
set GEN_BASE_ADDR 0x01000000
set MON_BASE_ADDR 0x01040000
set SYS_CTRL_LPBK_BASE_ADDR 0x01080000

-- 10G Ethernet/10G Base R Design Example Base Address
set 10GMAC_BASE_ADDR 0x00000000
set PHY_IP_BASE_ADDR 0x00040000
set MDIO_BASE_ADDR 0x00010000
set XGMII_LB_BASE_ADDR 0x00010200
set 10G_RX_FIFO_BASE_ADDR 0x00010400
set 10G_TX_FIFO_BASE_ADDR 0x00010600

Again, if they assign those manually or there is a tool for doing this automatically (like the "Assign Base Address" function in Qsys)?
Altera_Forum
Honored Contributor
14 years ago
Dear Dave,

I could make the board-to-board test run at receiver side without the "master_write_32" error.

The monitor logic has a base address and several offsets for statistics:

set MON_BASE_ADDR 0x01040000
set MON_RXPKTCNT_EXPT 0x00000000
set MON_RXPKTCNT_GOOD 0x00000004
set MON_RXPKTCNT_BAD 0x00000008
set MON_RXBYTECNT_LO32 0x0000000C
set MON_RXBYTECNT_HI32 0x00000010
set MON_RXCYCLCNT_LO32 0x00000014
set MON_RXCYCLCNT_HI32 0x00000018
set MON_RXCTRL_STATUS 0x0000001C
set GEN_MON_LPBK 0x00000028

I started generating and monitoring at both sides manually and almost at the same time. At RX side, the system console log shows that the monitor is unable to receive any packet. That means the above memory locations are not touched during the transmission. How can I explain and fix this? If you have any suggestion for debugging?

Here is how the traffic controller works. The traffic generator at transmitter generates packets of different types, with random length(<=1518) and random payload then sends them in burst to receiver. The monitor at receiver just checks number of received packets, how many with errors, how many of each types,... In case I have a board-to-board test, my next step is to check the transmission, namely to compare the difference of data sent at TX and data received at RX. But, I still don't know whether packet contents are temporary stored somewhere at both sides or not. How do you think about this plan?

Again, I need to understand the memory map. I could not find how the addresses are mapped and allocated in the design. I think this is a critical point for me now. Please help to give me a clue if possible.

Thanks.
Altera_Forum
Honored Contributor
14 years ago
--- Quote Start ---

Again, I need to understand the memory map. I could not find how the addresses are mapped and allocated in the design. I think this is a critical point for me now. Please help to give me a clue if possible.

--- Quote End ---

My advice: You have the code in front of you. Look through it and understand it.

If the example is too complicated, then stop looking at it, and work on something simpler. Go and listen to the SystemConsole webinar, go and create some simple designs that use SystemConsole, eg., the JTAG-to-Avalon-MM tutorial I wrote on the AlteraWiki. Basically, become comfortable with Qsys and Tcl. Then go back to the 10G problem.

Cheers,
Dave
Altera_Forum
Honored Contributor
14 years ago
--- Quote Start ---
My advice: You have the code in front of you. Look through it and understand it.
--- Quote End ---

Thanks for your advice. I am studying the code and learning System Console.
Altera_Forum
Honored Contributor
14 years ago
Dear Dave,

With my purpose of having a board-to-board 10GbE design, I have focused more in hardware test of the functionality of MAC or/and PHY layer. So, I have spent days to study this design (10G Ethernet and 10G Base-R PHY) and similar Altera reference designs such as:

10-Gbps Ethernet Reference:
http://www.altera.com/literature/ug/10g_ethernet_user_guide.pdf?gsa_pos=1&wt.oss_r=1&wt.oss=10-gbps%20ethernet%20reference%20design%20user%20guide

10-Gbps Ethernet MAC and XAUI PHY Interoperability Hardware Demonstration:
http://www.altera.com/literature/an/an638.pdf?gsa_pos=6&wt.oss_r=1&wt.oss=10-gbps%20ethernet%20reference%20design%20user%20guide

Triple Speed Ethernet Data Path Reference Design:
http://www.altera.com/literature/an/an483.pdf

Stratix II GX 10GbE Loopback Reference Design
http://www.altera.com/literature/an/an561.pdf

And I found out that all the reference/example designs provided by Altera only support several loopback tests, either internal or local (external) loopback. While board-to-board testing is also necessary and more difficult (my personal thinking), Altera do not provide any demo tests. And so far, I even could not find related tutorials anywhere. How should we understand this situation? If you can share your opinion or suggest me a way to go on with my target design.

Thank you very much.
Altera_Forum
Honored Contributor
14 years ago
--- Quote Start ---

And I found out that all the reference/example designs provided by Altera only support several loopback tests, either internal or local (external) loopback. While board-to-board testing is also necessary and more difficult (my personal thinking), Altera do not provide any demo tests.

--- Quote End ---
You can't expect Altera to provide a reference for everything, they provide just enough to get you started.

--- Quote Start ---

And so far, I even could not find related tutorials anywhere. How should we understand this situation? If you can share your opinion or suggest me a way to go on with my target design.

--- Quote End ---
The single-board reference design gives you everything you need to learn how to use the tools. You need to study the reference design until you can create that same design from scratch. Once you can do that, you have everything you need to create a dual-board design from scratch.

If you think your solution would be useful to others, then please write a tutorial and post it to the Altera wiki.

Cheers,
Dave

Forum Discussion

10G Ethernet and 10G Base-R PHY Hardware Demonstration Design

18 Replies

Recent Discussions

Agilex 5 FPGA EMIF: sharing LPDDR4 reference clock between Bank 3A and Bank 3B

Check COD for EP4CE10F17C8N

Unable to Generate Valid PAM4 Eye Diagram and BER Measurements on Stratix 10 SI E-Tile Native PHY

Cyclone IV GX Invalid JTAG ID Code - EPCS32

Functional fail and suspect relate to the component