DE10-Agilex: aocl diagnose fails with PCIe-to-fabric read error (0xffffffff)

## 1. Problem description

We are experiencing an issue with our **Terasic DE10-Agilex board** when running the Intel FPGA SDK for OpenCL diagnostic.

We have been following the **DE10-Agilex (revC) OpenCL BSP User Manual**, available on the [Terasic DE10-Agilex website](https://www.terasic.com.tw/cgi-bin/page/archive.pl?Language=English&CategoryNo=142&No=1252&PartNo=4).

After installing the required software, configuring the board, and setting up the PCIe driver, we execute:

```bash

aocl diagnose

```

The OpenCL ICD check passes, but the board diagnostic fails with the following error:

```text

Vendor: Terasic

MMD INFO : [aclde10_agilex0] PCIe-to-fabric read test failed, read 0xffffffff after 1 attempts

Failed Board name not available.

DIAGNOSTIC_FAILED

```

**This is the issue we are trying to resolve. We have not been able to get `aocl diagnose` to pass.**

Importantly, Linux successfully detects the FPGA through PCIe, and the corresponding kernel driver is loaded and bound to the device.

The FPGA appears in `lspci` as:

```text

01:00.0 Processing accelerators [1200]:

Altera Corporation Device [1172:35b4] (rev 01)

Kernel driver in use: aclpci_de10_agilex

Kernel modules: aclpci_de10_agilex_drv

```

The associated device node is also present:

```text

/dev/aclde10_agilex0

```

The PCIe memory BARs are allocated and enabled, and bus mastering is enabled.

Therefore, the FPGA is recognized by the operating system, but the Terasic MMD fails when attempting to communicate with the FPGA fabric.

## 2. Hardware and software environment

Our setup consists of:

- **FPGA board:** Terasic DE10-Agilex

- **Processor:** AMD Ryzen 9 7900X3D 12-Core Processor

- **Motherboard:** MSI MPG B850 EDGE TI WIFI (MS-7E62)

- **Operating system:** Ubuntu Linux

- **Linux kernel:** `5.15.0-139-generic`

- **FPGA software:** Intel Quartus Prime Pro 21.2

- **OpenCL software:** Intel FPGA SDK for OpenCL 21.2

- **OpenCL BSP:** Terasic `de10_agilex`

- **Board configuration:** `B2E2_8GBx4`

• **Connection:** PCIe, with JTAG access through Quartus Programmer

## 3. Troubleshooting performed

### 3.1. PCIe detection and link verification

We verified that the FPGA was detected through PCIe using:

```bash

lspci -nn

sudo lspci -vv -s 01:00.0

```

The FPGA reported:

```text

LnkCap: Speed 8GT/s, Width x16

LnkSta: Speed 2.5GT/s, Width x16

```

The upstream PCIe port advertised a maximum speed of 2.5 GT/s, which could explain the negotiated Gen1 speed.

The driver also reported an unsuccessful attempt to retrain the PCIe link to a higher generation.

Despite the FPGA being detected and the driver being bound, `aocl diagnose` continued to fail with the PCIe-to-fabric read error.

### 3.2. Motherboard, PCIe connection, and BIOS configuration

To investigate possible hardware compatibility or PCIe initialization problems, we performed several tests:

- Removed and reinstalled the DE10-Agilex board multiple times.

- Checked and reconnected the PCIe connection.

- Replaced the original motherboard with the MSI MPG B850 EDGE TI WIFI (MS-7E62), selected for compatibility with the board's PCIe requirements.

- Reviewed and modified the motherboard BIOS settings to meet the required PCIe configuration.

- Repeated the PCIe detection and OpenCL diagnostic tests.

These hardware and BIOS adjustments did not produce a successful `aocl diagnose` result.

### 3.3. Kernel driver verification

We checked the DE10-Agilex kernel driver using:

```bash

modinfo aclpci_de10_agilex_drv

```

The installed module was:

```text

aclpci_de10_agilex_drv.ko

```

The module's kernel version information matched the running Linux kernel.

We also verified that the driver was bound to the FPGA PCIe device and that the corresponding device node existed.

Therefore, the original diagnostic failure occurred despite the kernel driver being installed and active.

### 3.4. OpenCL runtime and MMD configuration

We inspected the OpenCL runtime configuration.

The OpenCL ICD was configured to load:

```text

libalteracl.so

```

The Terasic board configuration pointed to:

```text

libterasic_agilex_mmd.so

```

The OpenCL ICD check passed during `aocl diagnose`.

We also identified an older Cyclone V board configuration and disabled it to eliminate possible interference with the Terasic DE10-Agilex MMD.

However, the diagnostic continued to report:

```text

PCIe-to-fabric read test failed, read 0xffffffff after 1 attempts

```

### 3.5. FPGA programming through JTAG

We verified that the FPGA was accessible through JTAG using Quartus Programmer.

The FPGA was detected with JTAG ID:

```text

C341A0DD

```

We used the precompiled bring-up files provided with the Terasic DE10-Agilex BSP, including:

```text

top.sof

vector_add.aocx

host

```

We successfully programmed the FPGA through JTAG using the supplied `top.sof`.

However, the computer was rebooted immediately afterward, so we did not verify whether `aocl diagnose` passed while that JTAG-programmed image was still active.

We would appreciate clarification on whether the programming and initialization procedure described in the manual is sufficient to ensure correct PCIe-to-fabric communication.

### 3.6. Precompiled OpenCL example

We also attempted to run the precompiled vector-add example provided with the Terasic BSP.

The example failed with:

```text

CL_DEVICE_NOT_AVAILABLE

```

Additionally, `aocl list-devices` did not return a usable FPGA device.

This is consistent with the failure reported by `aocl diagnose`, where the OpenCL runtime cannot successfully access the FPGA through the Terasic MMD.

## 4. Summary

The main observations are:

- Linux successfully enumerated the DE10-Agilex FPGA as PCIe device `1172:35b4`.

- The `aclpci_de10_agilex` kernel driver was loaded and bound to the device.

- The FPGA device node existed.

- PCIe BARs were allocated and enabled.

- The OpenCL ICD check passed.

- The FPGA was accessible and programmable through JTAG.

- We reseated the board multiple times, replaced the original motherboard, and adjusted BIOS settings for PCIe compatibility.

- We verified the kernel driver and OpenCL runtime configuration.

- The precompiled OpenCL vector-add example failed with `CL_DEVICE_NOT_AVAILABLE`.

**Despite these checks and troubleshooting steps, `aocl diagnose` failed with:**

```text

MMD INFO : [aclde10_agilex0] PCIe-to-fabric read test failed, read 0xffffffff after 1 attempts

```

We suspect that the issue may involve communication between the PCIe endpoint and the FPGA fabric, but we have not identified its root cause.

## 5. Questions for the Terasic / Intel Agilex community

**1. PCIe-to-fabric read failure**

What could cause the Terasic MMD to return `0xffffffff` during the PCIe-to-fabric read test when Linux successfully detects the FPGA, enables its PCIe memory resources, and binds the appropriate kernel driver?

**2. FPGA configuration**

Could this error indicate that the FPGA configuration does not contain the expected PCIe-to-fabric interface or that the interface has not been initialized correctly?

Is there a specific FPGA configuration or initialization sequence required for `aocl diagnose` to pass?

**3. Diagnostic procedures**

Are there recommended tests, registers, driver logs, or debugging procedures to determine why the PCIe-to-fabric read fails?

**4. Official bring-up procedure**

We have followed the **DE10-Agilex (revC) OpenCL BSP User Manual** available on Terasic's website.

Are there additional steps required to establish communication between the host and the FPGA fabric beyond the procedures described in this manual?

**5. Software compatibility**

Are there known issues involving Intel FPGA SDK for OpenCL 21.2, the Terasic DE10-Agilex BSP, or the PCIe kernel driver that could cause this diagnostic failure?

Any guidance or recommended troubleshooting procedures would be greatly appreciated.

Our objective is to successfully complete `aocl diagnose` and establish reliable OpenCL communication with the DE10-Agilex FPGA.