Forum Discussion

raghad's avatar
raghad
Icon for New Contributor rankNew Contributor
1 month ago

Is Spatial IP ready for LLM / transformer inference?

I am using FPGA AI Suite 2026.1.1 (with the new spatial compiler). Most of the FPGA AI Suite handbook examples I see are classical CNN / vision flows (ResNet-style) on PCIe, hostless JTAG, and SoC.

Is transformer / LLM inference (attention layers, variable sequence lengths, large KV-cache activations, etc.) something we can target today with dla_compiler + Spatial IP, or is Spatial still aimed primarily at CNN-like graphs, or is custom RTL expected?

And if yes, are there any LLM examples, guides, recommended flows, or known limitations?

Thanks,

ai suite
general

3 Replies

  • DomiP_Altera's avatar
    DomiP_Altera
    Icon for New Contributor rankNew Contributor
    1 month ago

    LLM inference is not supported across any FPGA Ai suite releases as of now. It's in the works.
    The standard FPGA AI Suite workflow is not supported for LLM / Transformer models on any platform.

  • DomiP_Altera's avatar
    DomiP_Altera
    Icon for New Contributor rankNew Contributor
    1 month ago

    Currently FPGA AI Suite 2026.1.1 (with the new spatial compiler) doesn't support LLM / transformer inference.

    • raghad's avatar
      raghad
      Icon for New Contributor rankNew Contributor
      1 month ago

      Could you please clarify:

      • Is LLM inference unsupported across all FPGA AI Suite releases, or is this something being actively worked on for a future release?
      • Can I confirm that the standard FPGA AI Suite workflow is not supported for LLM / Transformer models on the Agilex 5 E-Series 065B Modular Development Kit?
        The workflow I am referring to is:

      Hugging Face model(optimum-intel) → OpenVINO IR (.xml + .bin) → architecture_optimizer → dla_compiler (Sequential flow) or Spatial Compiler → Integrate the generated FPGA AI Suite IP into a Quartus Prime project → Generate the bitstream and program it onto the Agilex 5 E-Series 065B board → Run inference using the FPGA AI Suite runtime (host application).

      • I understand this may not be a push-button process and could require significant modifications to the generated RTL — but is this workflow still considered a viable starting point for implementing LLM / Transformer inference on the Agilex 5 E-Series 065B?

      Thank you very much.