Forum Discussion
ContributorIntel FPGA AI Sutie Inference Engine
Is there any official documentation on the DLA runtime or inference engine for managing the DLA from the ARM side? I need to develop a custom application for running inference, but so far, I’ve only found the dla_benchmark (main.cpp) and streaming_inference_app.cpp example files. There should be some documentation covering the SDK. The only documentation that i found related with is the Intel FPGA AI suite PCIe based design example https://www.intel.com/content/www/us/en/docs/programmable/768977/2024-3/fpga-runtime-plugin.html
From what I understand, the general inference workflow involves the following steps:
- Identify the hardware architecture
- Deploy the model
- Prepare the input data
- Send inference requests to the DLA
- Retrieve the output data
42 Replies
RubenPadialHello @JohnT_Intel ,
Here it is: https://consigna.ugr.es/?s=download&token=095745fa-b5b5-4d04-a372-f7a08c626780
It is based on dla_benchmakr
JohnT_AlteraRegular Contributor
Hi,
Can you share me with your code or step so that I can try duplicating the issue from my side?
- JohnT_Altera
Hi,
You may make use of dla_benchmark apps and modfy from there. The new method should be as using "device_name.find("FPGA")"
- RubenPadial
Hello @JohnT_Intel
Taking dla_benchmark as an example, I get the following error:
[ ERROR ]
runtime/hps_packages/openvino/src/inference/src/ie_common.cpp:75
runtime/plugin/src/dlia_infer_request.cpp:53 Number of inference requests exceed the maximum number of inference requests supported per instance 5I'm looping the inference request because I need to instantiate the DLA and continuously request inferences with new data. Each inference must be a single request, so I set nireq=1 and niter=1. Once an inference is finished, I request a new one with new input data.
Therefore, I loop from step no. 9 to 11, obtaining the new input data before filling the blobs.
Is this approach correct? I understand a real application needs to instantiate de DLA and keep filling input to compute the CNN oputput with new data.
- JohnT_Altera
Hi Ruben,
Currently the only documentation is from the OpenVINO tools. When you are using HETERO:FPGA, CPU then the OpenVINO will try the AI in FPGA whenever it is possible and if it is not possible then the layer will be performed in CPU side. The OpenVINO will automatically communicate with the FPGA MMD driver
Let me know if you have further queries on this or you need any help on this.
- RubenPadial
Hello @JohnT_Intel ,
But when I use "GetAvailableDevices()" method I only get CPU as available device. There should be something I missed.
Form my point of view, there some points to be clarified from the Intel/Altera side to use OpenVINO tool in FPGA devices with FPGA AI Suite.
- JohnT_Altera
Hi Ruben,
Currently we do not have any document publish. Let me check internally if we have any documentation to share out.
- RubenPadial
Hello @JohnT_Intel ,
I know both example applications are based on OpenVINO runtime but I cannot find anything about FPGA and HETERO plugin to make inferences in HETERO:FPGA,CPU mode. This is the documentation I found https://docs.openvino.ai/archives/index.htmlI will very helpful any official documentation from Intel side to make Intel FPGA AI suite really useful.
