Forum Discussion
New ContributorData level parallelism on FPGA with kernel replication using oneAPI
Hi asenjo,
Sorry for late reply, I managed to consult one of my respective team member into your question. Based on your written code, the buffers go out of scope at the end of the VectorAdd() function and the kernels get serialized instead in running in parallel.
The main() would look something like this:
1. buffer a_buf1{a_vector.begin()+begin1, a_vector.begin()+end1};
2. buffer b_buf1{b_vector.begin()+begin1, b_vector.begin()+end1};
3. buffer sum_buf1{sum_parallel.begin()+begin1, sum_parallel.begin()+end1};
4.
5. buffer a_buf2{a_vector.begin()+begin1, a_vector.begin()+end2};
6. buffer b_buf2{b_vector.begin()+begin1, b_vector.begin()+end2};
7. buffer sum_buf2{sum_parallel.begin()+begin1, sum_parallel.begin()+end2};
8.
9. auto e0 = VectorAdd<true,0,2,4>(q, a_buf1, b_buf1, sum_buf1);
10. auto e1 = VectorAdd<true,1,2,4>(q, a_buf2, b_buf2, sum_buf2);
11. q.wait();
Another option is to use sub buffers:
Another option is to use USM but but then the user is responsible to copy data back and forth themselves:
https://www.intel.com/content/www/us/en/developer/articles/code-sample/vector-add.html
Thanks.
Regards,
Aik Eu
New ContributorThanks @aikeu !
On a related note, I'm now having an issue with this strategy of partitioning the buffers for each FPGA IP/kernel. In my example I'm using:
using IntVector = std::vector<int,oneapi::tbb::cache_aligned_allocator<int>>; IntVector sum_parallel(vector_size);
that as far as I know would align my vectors to 128Bytes boundaries. Actually, everything works fine if I use a buffer to refer to the whole IntVector:
buffer sum_buf{sum_parallel};
However, when partitioning sum_parallel in one buffer per kernel (with the corresponding block of data):
buffer sum_buf{sum_parallel.begin()+begin, sum_parallel.begin()+end};
I get this warning at runtime:
** WARNING: [aclbitt_s10mx_pcie0] NOT using DMA to transfer 8388608 bytes from device to host because of lack of alignment ** host ptr (0xd5198d0) and/or dev offset (0x3e0000000) is not aligned to 64 bytes
I then found how to pass an allocator to the buffer constructor:
buffer sum_buf{sum_parallel.begin()+begin, sum_parallel.begin()+end, oneapi::tbb::cache_aligned_allocator<float>{}};
But I still get the not-aligned warning and the corresponding slowdown when moving the data to/from the FPGA. I'm pretty sure that each "sub-buffer" (per kernel) is not aligned because I get 4 warnings when invoking 4 kernels or 16 warnings if I create 16 FPGA kernels. What am I doing wrong?
Thanks once again.
Rafa.
