Forum Discussion

Altera_Forum's avatar
Altera_Forum
Icon for Honored Contributor rankHonored Contributor
14 years ago

DDR3 instantiation, NIOS, Avalon, Qsys

Goal: try to instantiate DDR3 with the DIMM in the Stratix IV E kit (DDR3 with NIOS using Avalon-MM).

Goal: try to harness already existing DDR3 Stratix IV E kit work for a new project.

The kit comes with Micron MT18JSF25672AY-1G1D.

Overall: what do I do to instantiate a wide address DIMM + NIOS system?

Overall: how do I duplicate the DDR3 BTS? I cannot figure out answers to the questions below based on this example.

part 1

Consider a DIMM like the MT18JSF2567AY-1G1D (72 databits, 2 ranks, 18 total DRAMs). Example, UniPHY DDR3 chooses 14 row and 10 column address bits. With 2 ranks and 3 bank bits, seems like the total# of address bits is 29 (is this right?). I try to bring this memory into Qsys, get errors.

Error: System.cpu_to_sdram.m0: sdram.avl (0x0..0x7fffffff) is outside the master's address range (0x0..0x7ffffff).

The error seems to say the address space is outside the master’s range, which at 0x7ffffff is 27 bits. The master is either NIOS or TSE DMA.

Why is the master’s range only 27 bits, when Avalon-MM allows 32 bit addressing? Is there a limit on the number of bits of mapping address per device on an Avalon-MM network?

Instantiating wide address DDR3 DIMM, part 2

(see ddr3_attempt_2.qsys).

This is another error I get when trying to bring in the DDR3 DIMM. In this case, the address map for the DDR3 memory does not get assigned.

Error: System.nios2: integer value too large to represent

while executing

"format "0x%08x" $NUMBER "

invoked from within

"return [ format "0x%08x" $NUMBER ]"

(procedure "proc_num2hex" line 2)

invoked from within

"proc_num2hex $slave_info_array(start) "

invoked from within

"lappend slaves_start_address "[ proc_num2hex $slave_info_array(start) ]""

("foreach" body line 3)

invoked from within

"foreach slave_info $address_map_dec {

array set slave_info_array $slave_info

lappend slaves_start_address "[ proc_num2hex $slave_info_..."

(procedure "proc_get_address_map_slaves_start_address" line 5)

invoked from within

"proc_get_address_map_slaves_start_address $slave_map_param "

invoked from within

"lsort -ascii [ proc_get_address_map_slaves_start_address $slave_map_param "

invoked from within

"set slave_address [ lsort -ascii [ proc_get_address_map_slaves_start_address $slave_map_param ]]"

(procedure "proc_get_lowest_start_address" line 2)

invoked from within

"proc_get_lowest_start_address instSlaveMapParam "

invoked from within

"proc_num2hex [ proc_get_lowest_start_address instSlaveMapParam ] "

invoked from within

"set inst_master_paddr_base [ proc_num2hex [ proc_get_lowest_start_address instSlaveMapParam ] ]"

(procedure "sub_validate_check_module" line 14)

invoked from within

"sub_validate_check_module"

(procedure "validate" line 4)

invoked from within

"validate"

10 Replies

  • Altera_Forum's avatar
    Altera_Forum
    Icon for Honored Contributor rankHonored Contributor

    The Nios II core can only access up to 2GB of data memory and 256MB of instruction memory. So the limitation is the CPU and not the interconnect (which is limited to 4GB per master). If you look at the bitfield encoding of the jump instructions you'll see that it only supports a 26 bit encoding. So 26-bit * 4 bytes/word (instruction master only reads 32-bit words so the lower 2-bits of the address are redundant) equals a 256MB absolute address.

    The two easiest ways to address this limitation:

    1) Reduce the width of your memory which will in turn reduce it's span.

    2) Place the memory behind a windowed bridge. I have seen so many window bridges that one of them must be posted on the forum or alterawiki by now. If you don't find it let me know and I'll see if one is kicking around locally on my computer.
  • Altera_Forum's avatar
    Altera_Forum
    Icon for Honored Contributor rankHonored Contributor

    Thanks for your response.

    Reducing column width:

    Say I instantiate UniPHY DDR3 with a DIMM (like the Micron one

    that comes with Stratix IV E kit).

    And it's too large of a span, for reasons you mentioned.

    Can I keep everything exactly the same, with one exception: simply

    change the column bits from 10 to 8?

    (I did this, and all the errors listed above go away. Basically, I'm trying to

    reduce the usage of that DIMM by 4: only use the bottom 1/4th of the available columns).

    Is this okay, or will there be yet other issues by changing the column bits

    in instantiation from 10 to 8?

    Thanks,

    Aaron
  • Altera_Forum's avatar
    Altera_Forum
    Icon for Honored Contributor rankHonored Contributor

    I have done that in the past with the legacy controller and it worked fine. You will probably end up with an efficiency hit since you are only using part of a row this way (more overhead)

  • Altera_Forum's avatar
    Altera_Forum
    Icon for Honored Contributor rankHonored Contributor

    Ok, here's a follow up.

    I want 72 bit UDIMM + Qsys + UniPHY

    But we need to click power of 2 bus widths for SOPC and Qsys.

    HOW DO I USE QSYS AND UNIPHY TO DO 72 BIT DDR3, like for a DIMM?
  • Altera_Forum's avatar
    Altera_Forum
    Icon for Honored Contributor rankHonored Contributor

    Qsys doesn't support 9-bit symbols (bytes) so enabling 72-bit widths is not going to work. If you plan on sharing this memory with other masters I think a custom bridge would be your best bet. You can expose a subset of the memory span to the CPU and a wide and deep address span to other logic in your design. So the port for the CPU would be exposed to your Qsys system and the wide port would need to be handled outside the system.

    Now if you were planning to have Nios II access a 72-bit memory interface then I would stop right there since that doesn't make sense. Nios II has only 32-bit masters so going wider than that just for the CPU is overkill and a waste of logic. Processors typically never access the full width of a DIMM either, those extra 8 bits are reserved for ECC which is handled my the memory controller.
  • Altera_Forum's avatar
    Altera_Forum
    Icon for Honored Contributor rankHonored Contributor

    Thanks for replying.

    I will not need NIOS to access 72 bits.

    Mostly, I just want to get something to work, but I'm stuck with a 72 bit DIMM.

    If it's only 32 bits, that's okay. If it's 64 bits through two 32 bit accesses, that's okay.

    Right now I can compile, with Qsys => Quartus,

    with a 72 bit DDR3 Qsys UniPHY instantiation.

    Looks in the verilog code that the interface is 256 bits (64 * 4 based

    on DDR (dual data rate) and then AFI being a half rate interface).

    And I can download, and see some signals on signal tap.

    The main problem, simply: calibration fails!!!

    The interface doesn't work.

    Note: I'm using Stratix IV E eval kit board.

    This has a working DDR3 design (board test system).

    That design uses AltMemPHY, not UniPHY.

    I'm trying to match up all my Qsys UniPHY settings to the working

    AltMemPHY DDR3 design, but still the calibration fails in HW.
  • Altera_Forum's avatar
    Altera_Forum
    Icon for Honored Contributor rankHonored Contributor

    Okay, I got a lot further.

    Managed to get the EMIF toolkit up and running.

    I have a 72 bit interface, Qsys, UniPHY.

    The EMIF toolkit says bytes 0 through 7 calibrate just fine.

    Byte 8 doesn't. Not sure why.

    The numbers shown below suggest that the 9th byte is not on the edge

    of signal integrity, but rather somehow just not working at all.

    Note, I don't need 72 bits of data. But for example if I want 64,

    with one ECC byte, then I need to calibrate all 72 bgits.

    Again, what's the magic to get 72 bits all calibrated, UniPHY + Qsys + DDR3?

    +----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+

    | DQS Calibration Settings |

    +----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+

    +-----------+------------------------------+---------------------------+---------------------------------+---------------------------------+------------------------+--------------------------------+--------------------------+--------------------------+

    | DQS Group | DQS Input Delay Setting (D4) | DQ/DQS Output Phase (Deg) | DQS Output Delay 1 Setting (D5) | DQS Output Delay 2 Setting (D6) | DQS Enable Phase (Deg) | DQS Enable Delay Setting (T11) | OCT Delay 1 Setting (D5) | OCT Delay 2 Setting (D6) |

    +-----------+------------------------------+---------------------------+---------------------------------+---------------------------------+------------------------+--------------------------------+--------------------------+--------------------------+

    | 0 | 1 | 540 | 0 | 3 | 405 | 0 | 0 | 3 |

    | 1 | 2 | 585 | 0 | 3 | 90 | 0 | 0 | 3 |

    | 2 | 1 | 585 | 0 | 6 | 405 | 2 | 0 | 6 |

    | 3 | 1 | 630 | 0 | 4 | 90 | 2 | 0 | 4 |

    | 4 | 1 | 720 | 0 | 3 | 225 | 1 | 0 | 3 |

    | 5 | 0 | 765 | 0 | 3 | 225 | 2 | 0 | 3 |

    | 6 | 1 | 720 | 0 | 6 | 270 | 0 | 0 | 6 |

    | 7 | 0 | 765 | 0 | 4 | 270 | 1 | 0 | 4 |

    | 8 | 15 | 810 | 15 | 3 | 405 | 4 | 15 | 3 |

    +-----------+------------------------------+---------------------------+---------------------------------+---------------------------------+------------------------+--------------------------------+--------------------------+--------------------------+

    +--------------------------------------------------------------------------+

    | DM Calibration Settings |

    +--------------------------------------------------------------------------+

    +--------+--------------------------------+--------------------------------+

    | DM Pin | DM Output Delay 1 Setting (D5) | DM Output Delay 2 Setting (D6) |

    +--------+--------------------------------+--------------------------------+

    | 0 | 2 | 0 |

    | 1 | 4 | 0 |

    | 2 | 5 | 0 |

    | 3 | 4 | 0 |

    | 4 | 2 | 0 |

    | 5 | 2 | 0 |

    | 6 | 4 | 0 |

    | 7 | 4 | 0 |

    | 8 | 15 | 0 |

    +--------+--------------------------------+--------------------------------+
  • Altera_Forum's avatar
    Altera_Forum
    Icon for Honored Contributor rankHonored Contributor

    Hi,

    in altmemphy (not uniphy) the no. of bits for the local address is calculated for half rate controller as bellow:

    For one chip select: width = row bits + bank bits + column bits – 2

    For multiple chip selects: width = chip bits + row bits + bank bits + column bits – 2

    you should be able to find this information in the controller handbook. the width is different if the controller is working on full rate.

    hope this helps
  • Altera_Forum's avatar
    Altera_Forum
    Icon for Honored Contributor rankHonored Contributor

    I'm not sure what could be happening to cause the ECC bits to not calibration since I have not used ECC before. One thing to check is whether the memory DIMM even has a 72-bit data bus. I could see memory module manufactures leaving the extra 8-bits out to save money in the DIMMs for PCs (i.e. not servers) since typically only server grade processors implement ECC. So calibration could be failing due to a lack of there being ECC memory present.

    The easiest way to check this is count the number of SDRAM ICs on the module, if it has 8 or 16 then you only have a 64-bit data bus.
  • Altera_Forum's avatar
    Altera_Forum
    Icon for Honored Contributor rankHonored Contributor

    --- Quote Start ---

    Goal: try to instantiate DDR3 with the DIMM in the Stratix IV E kit (DDR3 with NIOS using Avalon-MM).Goal: try to harness already existing DDR3 Stratix IV E kit work for a new project.The kit comes with Micron MT18JSF25672AY-1G1D. Overall: what do I do to instantiate a wide address DIMM + NIOS system?Overall: how do I duplicate the DDR3 BTS? I cannot figure out answers to the questions below based on this example. part 1 Consider a DIMM like the MT18JSF2567AY-1G1D (72 databits, 2 ranks, 18 total DRAMs). Example, UniPHY DDR3 chooses 14 row and 10 column address bits. With 2 ranks and 3 bank bits, seems like the total# of address bits is 29 (is this right?). I try to bring this memory into Qsys, get errors. Error: System.cpu_to_sdram.m0: sdram.avl (0x0..0x7fffffff) is outside the master's address range (0x0..0x7ffffff). The error seems to say the address space is outside the master’s range, which at 0x7ffffff is 27 bits. The master is either NIOS or TSE DMA. Why is the master’s range only 27 bits, when Avalon-MM allows 32 bit addressing? Is there a limit on the number of bits of mapping address per device on an Avalon-MM network? Instantiating wide address DDR3 DIMM, part 2 (see ddr3_attempt_2.qsys). This is another error I get when trying to bring in the DDR3 DIMM. In this case, the address map for the DDR3 memory does not get assigned. Error: System.nios2: integer value too large to represent while executing"format "0x%08x" $NUMBER " invoked from within"return [ format "0x%08x" $NUMBER ]" (procedure "proc_num2hex" line 2) invoked from within"proc_num2hex $slave_info_array(start) " invoked from within"lappend slaves_start_address "[ proc_num2hex $slave_info_array(start) ]"" ("foreach" body line 3) invoked from within"foreach slave_info $address_map_dec { array set slave_info_array $slave_info lappend slaves_start_address "[ proc_num2hex $slave_info_..." (procedure "proc_get_address_map_slaves_start_address" line 5) invoked from within"proc_get_address_map_slaves_start_address $slave_map_param " invoked from within"lsort -ascii [ proc_get_address_map_slaves_start_address $slave_map_param " invoked from within"set slave_address [ lsort -ascii [ proc_get_address_map_slaves_start_address $slave_map_param ]]" (procedure "proc_get_lowest_start_address" line 2) invoked from within"proc_get_lowest_start_address instSlaveMapParam " invoked from within"proc_num2hex [ proc_get_lowest_start_address instSlaveMapParam ] " invoked from within"set inst_master_paddr_base [ proc_num2hex [ proc_get_lowest_start_address instSlaveMapParam ] ]" (procedure "sub_validate_check_module" line 14) invoked from within"sub_validate_check_module" (procedure "validate" line 4) invoked from within"validate"

    --- Quote End ---

    Hi,Did your question have resolved.I am encountered the same problem,and let me referred it.