NDrange, work-itme level parallelism vs work-group level parallelism

Maybe I simplified things a little bit too much. It is not just about divisibility. As you said, there is no guarantee that work-groups running in different CUs would finish at the same time, hence some CUs will always remain unused. However, with more work-groups, the chance of a CU being unused will get smaller, resulting in closer-to-linear speed-up with number of CUs. Furthermore, at least based on what Altera's report claims, there is also work-group pipelining in place and hence, there could be multiple work-groups in-flight in the same CU at the same time and having more work-groups will further help to keep the CU busy.