Intel FPGA AI Sutie Inference Engine

Hello @JohnT_Intel ,

I know both example applications are based on OpenVINO runtime but I cannot find anything about FPGA and HETERO plugin to make inferences in HETERO:FPGA,CPU mode. This is the documentation I found https://docs.openvino.ai/archives/index.html

I will very helpful any official documentation from Intel side to make Intel FPGA AI suite really useful.

Hello @JohnT_Intel ,

But when I use "GetAvailableDevices()" method I only get CPU as available device. There should be something I missed.
Form my point of view, there some points to be clarified from the Intel/Altera side to use OpenVINO tool in FPGA devices with FPGA AI Suite.

Hello @JohnT_Intel

Taking dla_benchmark as an example, I get the following error:

[ ERROR ]

runtime/hps_packages/openvino/src/inference/src/ie_common.cpp:75
runtime/plugin/src/dlia_infer_request.cpp:53 Number of inference requests exceed the maximum number of inference requests supported per instance 5

I'm looping the inference request because I need to instantiate the DLA and continuously request inferences with new data. Each inference must be a single request, so I set nireq=1 and niter=1. Once an inference is finished, I request a new one with new input data.

Therefore, I loop from step no. 9 to 11, obtaining the new input data before filling the blobs.

Is this approach correct? I understand a real application needs to instantiate de DLA and keep filling input to compute the CNN oputput with new data.

Hello @JohnT_Intel,
Here it is: https://consigna.ugr.es/?s=download&token=0fcf80b0-8ff5-47da-8da7-5b9acebf1646

As you can see with the debug lines I included, program fails in line "inferRequestsQueues.push_back(std::move(
std::unique_ptr<InferRequestsQueue>(new InferRequestsQueue(*exeNetwork, nireq))));" in the 6th iteration.

Hello @JohnT_Intel,

I'm currently using FPGA AI Suite 2023.2 and OpenVINO 2022.3.1. I know it is no the latest rlease of FPGA AI Suite but I cannot move the project to FPGA AI Suite 2024.3 at this moment.

Hello @JohnT_Intel ,

Same grpah with nireq and niter set to 1 in every inference.

This is how I run the application:
./ris_app \
-arch_file=$arch \
-cm=$model \
-plugins_xml_file=$plugins \
-nireq=1 \
-niter=1 \
-d=HETERO:FPGA,CPU

As far as I know, I only used the graph once to configure the DLA, and then I continually request inferences to that instance. At least that was the objective.

Hello @JohnT_Intel ,

No, I run it once. In the application there is a loop from step 9 to 11. In the 6th iteration of the loop program fails.

Prior stepts to 9 are intended to configure the dla and create the dla instance. The aim of looping steps 9 to 11 is to continually request inference to the already configured dla.

Hello @JohnT_Intel ,

Yes, that's what I supposed. How should it be handled?

The inferRequest->wait(), inferRequest->startAsync() and inferRequestsQueue->waitAll() statements are used, and the output is properly retrieved, so the inference is completed. I don't know what happens with the request or how to handle/wait/stop the request once inference is finished.