CL_FLUSH is acting as Blocking call.

In that case set memory transfer to non-blocking as you have already done and enqueue the kernel, but use an event on the kernel enqueue and avoid using clFlush/clFinish after that. Then, continue with your other processing in the host and whenever it was required, "wait" on the event from the kernel enqueue (clWaitForEvents) to make sure the kernel has finished processing and then transfer the output data from the FPGA to the host.