Forum Discussion

rsing108's avatar
rsing108
Icon for Occasional Contributor rankOccasional Contributor
5 years ago
Solved

HBM2 interfacing with the PCI express hard IP STRATIX 10 MX

Hi,

I am trying to interface HBM2 with pci express hard ip+ on stratix 10 MX. I am referencing to this example design https://fpgacloud.intel.com/devstore/platform/18.1.0/Pro/pci-express-gen3-x16-avmm-dma-with-hbm2-memory-reference-design/?wapkw=hbm2%20interface%20pci%20express , but in this I am not able to understand that how the BAM master address width is 30 bits whereas when I am generating the same IP inside my qsys system for the same configuration I am getting 16 bits as the address width. The BAR size in both of the cases is 16 bits.

Is there any way I can manually edit this?

  • BoonT_Intel's avatar
    BoonT_Intel
    5 years ago

    Hi, Sorry for the mistake.

    I tried the same and I only can see the 16bits on the address. These address bits are affected by the setting of "Address width of bursting master" of the PCIe IP. I believe the developer hack the IP and change the address width to 30 instead.

    If you need, you can change this by modifying the g3x16_hbm2_avmm_bridge_512_0.ip file.

    Search for Address width of Bursting Master and you can change the value to 30.


22 Replies

  • BoonT_Intel's avatar
    BoonT_Intel
    Icon for Frequent Contributor rankFrequent Contributor
    5 years ago

    Hi, Are you getting 403 access denied? I tried the same and get this error. I think the link should work, let me check with the internal team. From your side, maybe can see if there any link that you can click to "contact intel" and report to IT from external. Thanks.


    • rsing108's avatar
      rsing108
      Icon for Occasional Contributor rankOccasional Contributor
      5 years ago

      Hi @BoonT_Intel , I am also getting the same 403 error. If you could get me the link that would be great. Meanwhile I will also try to file an intel IT request.

      Thanks.

  • rsing108's avatar
    rsing108
    Icon for Occasional Contributor rankOccasional Contributor
    5 years ago

    One difference I see is that the reference design's device is 1sm21bhu2f53e2vgs1 whereas my device is 1sm21chu2f53e2vg. But I do not think this should create any problem.

  • BoonT_Intel's avatar
    BoonT_Intel
    Icon for Frequent Contributor rankFrequent Contributor
    5 years ago

    Hi Sir,

    Yes, the BHU and CHU is the difference, for BHU device, the HBM2 is 8G and CHU device, CHU is 16G. So you see the address bits is 2x diff.

    1sm21chu2f53e2vg


    • rsing108's avatar
      rsing108
      Icon for Occasional Contributor rankOccasional Contributor
      5 years ago

      Thanks @BoonT_Intel for the reply. The difference in size is for the BAM_master address of PCIE IP and not the HBM ip. Also the CHU one has 30 bit address in the reference design and chu has 16 bit address for the same PCIE hard ip+ configuration.

      • rsing108's avatar
        rsing108
        Icon for Occasional Contributor rankOccasional Contributor
        5 years ago

        please note the typo in my last comment: "Also the BHU one has 30 bit address in the reference design and CHU has 16 bit address for the same PCIE hard ip+ configuration."

  • BoonT_Intel's avatar
    BoonT_Intel
    Icon for Frequent Contributor rankFrequent Contributor
    5 years ago

    Hi, Sorry for the mistake.

    I tried the same and I only can see the 16bits on the address. These address bits are affected by the setting of "Address width of bursting master" of the PCIe IP. I believe the developer hack the IP and change the address width to 30 instead.

    If you need, you can change this by modifying the g3x16_hbm2_avmm_bridge_512_0.ip file.

    Search for Address width of Bursting Master and you can change the value to 30.


    • rsing108's avatar
      rsing108
      Icon for Occasional Contributor rankOccasional Contributor
      5 years ago

      Thanks @BoonT_Intel , it indeed appears that this a some kind of a hack by the developer. Thanks! One a side note, can you refer me to maybe an example design/reference design where I can understand the working/usage of multiple BARS?

  • BoonT_Intel's avatar
    BoonT_Intel
    Icon for Frequent Contributor rankFrequent Contributor
    5 years ago

    Hi Sir,

    I thought the AN881 is already a multiple BAR design?

    It can access DDR4 and HBM2.

    BAR 2 for HBM2 and BAR4 for DDR4.

    By the way, you can check the design store and see if any design meet your needs - https://fpgacloud.intel.com/devstore/

    Thanks.


    • rsing108's avatar
      rsing108
      Icon for Occasional Contributor rankOccasional Contributor
      5 years ago

      Hi @BoonT_Intel ,

      A881 in the documentation uses multiple bars but the link to reference design is broken. The reference design I mentioned earlier uses only 1 bar i.e. for HBM.

      Thanks

  • BoonT_Intel's avatar
    BoonT_Intel
    Icon for Frequent Contributor rankFrequent Contributor
    5 years ago

    Great and thanks. If anyone of us gets the response/link fixed, we can come back here to give an update.


  • BoonT_Intel's avatar
    BoonT_Intel
    Icon for Frequent Contributor rankFrequent Contributor
    5 years ago

    Hi Sir,

    They told me the design file is corrupted in the server. Now they're trying to approach the design owner to get back the same file.

    I will get back to you immediately after I get the file. Very sorry for the inconvenience caused.


  • BoonT_Intel's avatar
    BoonT_Intel
    Icon for Frequent Contributor rankFrequent Contributor
    5 years ago

    Hi Sir,

    Are you able to make any progress there?

    Perhaps, I can get the design by end of this month. Thanks.


    • rsing108's avatar
      rsing108
      Icon for Occasional Contributor rankOccasional Contributor
      5 years ago

      Hi @BoonT_Intel ,

      I am currently stuck. It would be great if you could get me the design. Also, if possible can you get this design for 1sm21chu rather than 1sm21bhu? if not then 1sm21bhu is also fine.

      Thanks for your help.

  • BoonT_Intel's avatar
    BoonT_Intel
    Icon for Frequent Contributor rankFrequent Contributor
    5 years ago

    Hi Sir,

    I get the design zip file. However, the design is not yet validated. They will only publish the design after re-validation.

    However, I believe it will be helpful for you for reference purpose. Thus, please note that the design is provided AS-IS basis only.


    The file is too big, thus I can’t upload it here. I share the file with you using the other channel.

    Here is the step to download the file:-

    Download the filezilla and install - https://filezilla-project.org/download.php?type=client


    After install, open filezilla and login using -

    Host: secureftp.intel.com

    Username: tmp01798

    Password: Hello12!

    Port:21


    You will see and file name AN881.zip at the bottom right corner.

    right click the file and download.


    • rsing108's avatar
      rsing108
      Icon for Occasional Contributor rankOccasional Contributor
      5 years ago

      Hi @BoonT_Intel ,

      Thanks for the reference design. I changed the device to 1sm21chu2f53e2vg which is the device I have along with the HDL changes required and PIN assignments. I have not changed anything else. But I am getting this warning now.

      Warning(332049): Ignored create_clock at pcie_to_hbm2_ddr4_top.sdc(2): Incorrect assignment for clock. Source node: REFCLK_GXBL1C_CHTP already has a clock(s) assigned to it. Use the -add option to assign multiple clocks to this node. Clock was not created or updated.
      Info(332050): create_clock -name {REFCLK_GXBL1C_CHTP} -period "100Mhz" [get_ports {REFCLK_GXBL1C_CHTP}]

      REFCLK_GXBL1C_CHTP is the iopll reference clock for HBMC. Can you tell me what can be the reason for this as according to me it is just defined in the project sdc file once.

      • rsing108's avatar
        rsing108
        Icon for Occasional Contributor rankOccasional Contributor
        5 years ago

        Hi @BoonT_Intel, please I have been working on the reference design that you provided. There are a few bugs that I found in one of the files but the design is still not working. I am attaching the updated file here. I suspect there are other problems with it too.

        Can you please tell me when will the final design be posted after validation? Hoping that all the bugs will be removed by then.

        Also, can I do something to accelerate this process?

        Thanks.