Poor heterogeneous memory performance in Quartus 14.1

Have you tried using the CL_MEM_HETEROGENEOUS_ALTERA flag? This hint tells the host runtime to allocate into the heterogeneous memory. What you may be seeing is the host copying out of default memory prior to kernel execution, and that overhead is getting lumped into the kernel execution time.