TiensungA_Altera
New ContributorJoined 4 years ago
User Profile
New Contributor
User Widgets
Contributions
Re: Routing PL DDR to PS
hi, Are you specifically asking to do the following? HPS CPU (ARM) <--AXI Bridge--> FPGA Interconnect <---> DDR Controller (in FPGA fabric) <---> DDR chips (physically connected to FPGA pins) If so, it could be possible to get the HPS CPU to access the DDR connected to the PL. We need to have some RTL magic/implementation to expose the memory controller interface in the FPGA as a slave memory device and then, configure the HPS to use it as the system memory. There lies many challenges that I can't confirm. We may have coherency issues (IO Cache coherency but the SCU anyways does not support IO Cache coherency from other initiators or DMA agents). And also performance issues like higher memory latencies. Since the memory interface is routed through the HPS to FPGA bridge, HPS running an OS like Linux which requires MMU and Cache enablement may run into coherency issues. UBOOT SPL or bare-metal running on HPS could run without enabling DCache/ICache. Having said that, if it is just the HPS CPU cores which relies on the SCU, this could be possible as the memory coherency between the cores are maintained by SCU and the cache managment may be fine. You will also have to ensure that, no other agents in FPGA or DMA devices writes to the DDR/memory (This could be solvable easily by SW intervention, SW cache coherency management). From a SW standpoint, you have to modify the UBOOT/Linux device tree to use the memory mapped addresses at the HPS to FPGA bridge. The FPGA has to be configured before starting the HPS. *Add-on on SW, the UBOOT SPL SDRAM device driver or module may need to be altered/hacked to bypasss the default drivers which runs the hard memory controller in A10. Else it will throw an error during boot-up. This is the tentative reply to your question. A readily available workaround does not exist today as it may requires some POC or exploration by your team. We will consult more experts and if we have anything extra or corrections, we will reply later to this discussion thread. ThanksRe: Operating system kernel-level FPGA bridge communication
hi, By right, the devmem2 must be used to read/write to the HPS2FPGA bridge using the mapping below for Agilex5. I can see that your mapping is incorrect. If you are using the LW HPS2FPGa, it starts at 0x1FC0_0000. You do not need to create any nodes in the device tree. The nodes in the device tree are only required if you are writing/using SOFT IP Device drivers in Linux kernel to access the bridge. Without drivers and with user-space interface, you should be able to use devmem2. The bridges will be enabled by default if you do a FPGA configuration in Linux. If you perform this in U-boot, you will have to do a bridge enable in uboot before booting the Linux. The configs that you use above example, CONFIG_FPGA_BRIDGE and _REGION are only required if you are doing FPGA configuration in Linux as they are required by the FPGA manager device driver "stratix10_soc.ko" driver to configure the 2nd stage FPGA core.rbf. In short, the bridges should be enabled if you have configured the FPGA in UBOOT prior to loading Linux.Re: Agilex 5 EMAC to EMAC : Driver error
I would be questioning why emac1 and emac0 clk are disabled for some unknown reason. From the CLK dump from Linux, I believed the Quartus design may have an issue. These 2 clock gates the GMAC0 and GMAC1 (emac0_en , emac1_en). That is why GMAC2 is the only one working. We don't know why yet. Thanks for all the dumps. Can you send us the file below? We would need to inspect your SOF file. XXX.sofRe: Agilex 5 EMAC to EMAC : Driver error
On top of what my other colleagues asked, Can you also dump, the boot Linux log for all the lines that mentions "socfpga-dwmac"? I want to understand if the Hard IP is even accessible or enabled correctly by the Quartus design for Ethernet. The example i have here is for gmac2. But you are using gmac0 and gmac1. It seems you are unable to even perform the initial setup of the ETH HW which has a DMA engine. This requires write to the ETH HW CSR register address. I want to see if the GMAC0 and GMAC1 are even enabled correctly from the Quartus design at the HPS. In the Quartus HPS config, did you enable GMAC0 and GMAC1 from the dropdowns to select the function? GMAC0: → [ RGMII ] / [ SGMII ] / [ Disabled ] GMAC1: → [ RGMII ] / [ SGMII ] / [ Disabled ] This should be propagated as a handoff data to the U-boot HPS FSBL which will configure the PINMUX and also CLK settings. The Handoff is important to the HPS to configure the ETH clock settings and pinmux. The GMAC0 and GMAC1 reset should already been de-asserted by the U-boot during boot-up. So, i don't suspect the reset de-assertion. It is either the CLK or the PINMUX or a combination of both. Without CLK, the IP will be permanently be disabled. CLK is the major suspect. In short, I suspect that, the GMAC0 and GMAC1 CLK are disabled by a bad Quartus configuration. Log for Linux: Example for a working GMAC on GMAC2. I need to see the similar log for GMAC0 and GMAC1 on your end. I want to know how far it went before it threw the error. [ 0.744790] socfpga-dwmac 10830000.ethernet: Adding to iommu group 0 [ 0.746937] socfpga-dwmac 10830000.ethernet: IRQ eth_wake_irq not found [ 0.747646] socfpga-dwmac 10830000.ethernet: IRQ eth_lpi not found [ 0.748310] socfpga-dwmac 10830000.ethernet: IRQ sfty not found [ 0.749186] socfpga-dwmac 10830000.ethernet: SMTG Hub Cross Timestamp supported [ 0.750124] socfpga-dwmac 10830000.ethernet: User ID: 0x76, Synopsys ID: 0x31 [ 0.750897] socfpga-dwmac 10830000.ethernet: XGMAC2 [ 0.751426] socfpga-dwmac 10830000.ethernet: DMA HW capability register supported [ 0.752217] socfpga-dwmac 10830000.ethernet: RX Checksum Offload Engine supported [ 0.753010] socfpga-dwmac 10830000.ethernet: COE Type 1 [ 0.753568] socfpga-dwmac 10830000.ethernet: TX Checksum insertion supported [ 0.754310] socfpga-dwmac 10830000.ethernet: TSO supported [ 0.754904] socfpga-dwmac 10830000.ethernet: Enabled L3L4 Flow TC (entries=16) [ 0.755669] socfpga-dwmac 10830000.ethernet: Enabled RFS Flow TC (entries=10) [ 0.756422] socfpga-dwmac 10830000.ethernet: TSO feature enabled [ 0.757061] socfpga-dwmac 10830000.ethernet: SPH feature enabled [ 0.757698] socfpga-dwmac 10830000.ethernet: Using 40/40 bits DMA host/device widthRe: Linux UIO IRQ related periodic CPU usage
We have tried several experiments, and it seems it is unrelated to the interrupt storm. If you have ran out of ideas, my only last resort is not use UIO and write a simple Kernel driver that maps to the IRQ. Just creates a simple handler ISR to dump out the number of interrupts per minute in printf. Every time, you receive an interrupt, you increase a counter. You poll this counter and you do a printf every 1-2minutes so, you don't clutter your log terminal. By bypassing UIO, we can definitely check isolate where the problem is? If there are no CPU spikes by doing so, then, the UIO is a suspect. To me, this would be the simplest way to find out which component causes the problem.Re: Linux UIO IRQ related periodic CPU usage
Item 4: Still some question on the user application design: Just to double confirm, WRITE() must always be after READ() and only after you are done with any data processing/event handling. In the interrupt handler, any interrupt trigger will automatically disable the IRQ. The READ() will get a wakeup and at this point, the IRQ is disabled. The call to write() will re-enable the IRQ. If you don't do a WRITE at all, do you see the CPU spikes? In this scenario, the IRQ should be disabled and not entertaining edge interrupt triggers. Check "cat /proc/interrupts" to see if the counts stopped. And if you do a WRITE() from your test, Check "cat /proc/interrupts" to see if the counts increment as expected or it storms with a huge number. bool wait_for_irq (TX_IRQ_HANDLE_S* pHandle) { if (!pHandle || pHandle->txTrigIrqFd < 0) { std::cout << "ERROR: handle"; return false; } uint32_t info = 1; ssize_t nb = write(pHandle->txTrigIrqFd, &info, sizeof(info)); <- this re-enables IRQ, should always be only when you are ready to for the next event. if (nb != (ssize_t)sizeof(info)) { std::cout << "ERROR: writing"; return false; } nb = read(pHandle->txTrigIrqFd, &info, sizeof(info)); if (nb == (ssize_t)sizeof(info)) { return true; } return false; }Re: Linux UIO IRQ related periodic CPU usage
2. Yes, I have done it before, it works, but I need the more stricter timing with IRQ. >The only clue I have is that, it seems the interrupt handling may be the culprit. Even though it is edge triggered, apparently the interrupt did not get cleared properly and results in an interrupt storm. Can you check if the Linux UIO kernel driver is receiving the interrupt storm and hence causing the CPU spike? Probably using "cat /proc/interrupts" to see the interrupt statistics. You mentioned the FPGA "The installed FPGA image is doing nothing more than making interrupts to an UIO device, 50 times a sec." Is this part of the Arria10 GHRD? Can you share us more information on the design? DT you shared: tx_trig_irq { /* /dev/uio4 */ compatible = "test_irq", "generic-uio"; interrupts = < 0 0x16 IRQ_TYPE_EDGE_RISING >; interrupt-parent = <&intc>; }; You are using 0x16 = 22 , so, the IRQ = 22 + 32 = 55 (F2S_FPGA_IRQ4). This seems right. The FPGQ IRQs for Arria10 begins 51 with https://www.intel.com/content/www/us/en/docs/programmable/683711/21-2/gic-interrupt-map-for-the-arria-10-soc-hps.html 51 FPGA F2S_FPGA_IRQ0 - Level or Edge 52 FPGA F2S_FPGA_IRQ1 - Level or Edge 53 FPGA F2S_FPGA_IRQ2 - Level or Edge 54 FPGA F2S_FPGA_IRQ3 - Level or Edge 55 FPGA F2S_FPGA_IRQ4 - Level or Edge 56 FPGA F2S_FPGA_IRQ5 - Level or Edge 57 FPGA F2S_FPGA_IRQ6 - Level or EdgeRe: Unable to access the entire 4GB DDR from Agilex 7 HPS
Can you try the change below? This change maps to the SDRAM upper > 2GB to 0x1_0000_0000. Technically, 0x1_0000_0000 will map back to 0-2G. So, we should use 0x1_8000_0000 for 2-4G. memory { reg = < 0 0x00000000 0 0x80000000>, <1 0x80000000 0 0x80000000>; }; ThanksRe: SDM Mailbox access in intel agilex 5
Yes, that is correct. ATF is the middleman for all the mailbox communication to SDM. The HPS HWMON software driver sends the SMC Call to ATF when then sends the HWMON READVOLT to SDM. Make sure your Linux OS build has CONFIG_SENSORS_SOC64=y. On Linux check if the HWMON driver is probed successful root@agilex:/# dmesg | grep "soc64" [ 1.268886] soc64_hwmon_probe [ 1.271949] soc64_hwmon_probe Initialized 1 temperature and 6 voltage channels [ 1.271957] soc64-hwmon soc:firmware:hwmon: Initialized 1 temperature and 6 voltage channels Read the temperature or voltages like this ~# cat /sys/devices/platform/soc:firmware:svc/soc:firmware:svc:hwmon/hwmon/hwmon1/temp1_input 78125 ~# cat /sys/devices/platform/soc:firmware:svc/soc:firmware:svc:hwmon/hwmon/hwmon1/temp2_input 82125 ~# cat /sys/devices/platform/soc:firmware:svc/soc:firmware:svc:hwmon/hwmon/hwmon1/temp3_input 80875 :~# cat /sys/devices/platform/soc:firmware:svc/soc:firmware:svc:hwmon/hwmon/hwmon1/temp4_input 76375
