Forum Discussion

Altera_Forum's avatar
Altera_Forum
Icon for Honored Contributor rankHonored Contributor
12 years ago

About Altera OpenCL Compilation

Hello everyone,

I am just wondering if anyone know the difference between compiling kernels in Linux or Windows (in term of run-time performance and total compilation time). I know i could use .aocx file compiled from windows to run in Linux, but there seems to be a difference in performance sometimes. I am trying to figure out which OS is better for compilation.

Also, I am wondering if kernels that utilize task parallelism (1 workitem per workgroup) usually takes longer and uses more memory to compile than kernels utilizing data parallelism (many workitems per workgroup).

In addition, I am just wondering what is the latency to access the local memory in term of clock cycles; or is the latency heavily dependent on the size of local memory/how the memory is accessed?

Thanks!

Ryan

27 Replies

  • Altera_Forum's avatar
    Altera_Forum
    Icon for Honored Contributor rankHonored Contributor
    11 years ago

    Hi,

    I do not have any data on Windows vs. Linux Quartus compilation times. This could already be mentioned in other Altera forums.

    Quartus may behave slightly differently (place/route) between platforms. This explains why you are sometimes seeing different aocx performances. I think there is no clear winner platform; the difference is more like running the same algorithm with different seeds. You can again refer to other forums for possible more detailed answers.

    When it comes to the aoc compilation, my observation is that Linux could be faster just because gcc does a better job. Otherwise, aoc compiler should behave exactly the same way on Linux and on Windows, i.e. you will obtain the same Verilog output for the same input file.

    Compilation flow is mostly the same whether the kernel is a task or an ndrange. The most significant difference stems from loop-carried memory dependences. If your task kernel has significant number of memory operations that have memory dependences carried by loops, computing these dependences and generating the appropriate hardware may take some time. Eliminating these dependences will reduce the compile-time as well as may increase performance.

    The latency of local memory accesses mainly depends on the complexity of the local memory system that is generated. If a load/store instruction is the only instruction that is connected to a local memory port, its latency will be minimal. However, if multiple local memory accesses are going to the same port then the latency will be increased to account for this "competition" (i.e. interconnect). You can refer to the "Optimizing Local Memory Accesses" Section in the "Best Practices Guide".
  • Altera_Forum's avatar
    Altera_Forum
    Icon for Honored Contributor rankHonored Contributor
    11 years ago

    Thank you so much for the help! I just have one quick question however: the AOCL programming guide says that the number of access to local memory should be less or equal to 4 for best efficiency. I am just wondering if this means I shouldn't access the local memory 4 times concurrently, or does this mean in the entire kernel I should only have 4 or less different local memory access in total?

    Regards,

    Ryan
  • Altera_Forum's avatar
    Altera_Forum
    Icon for Honored Contributor rankHonored Contributor
    11 years ago

    It is the latter; the programming guide recommends that "the entire kernel should have 4 or fewer different accesses to local memory.

    Basically, each load/store instruction in the kernel becomes a client (i.e. master) for local memory. Because local memory has at most 4-ports, if you have 4 load/store instructions, each port will be connected to a single load/store so that load/store instructions will not compete with each other. This guarantees the most efficient hardware.

    If you have 3 or fewer store instructions and many loads (loads+stores > 4), the compiler may choose to replicate the local memory. This also gives fast accesses at the local memory at the expense of RAMs.

    Further, the compiler performs some optimizations to partition the local memory based on the access patterns. Hence, even if there are more than 4 store instructions, you may still get efficient hardware. However, this depends on the complexity of your access patterns and may not always be possible.
  • Altera_Forum's avatar
    Altera_Forum
    Icon for Honored Contributor rankHonored Contributor
    11 years ago

    Thank you for the clarification! I am just wondering if I have a few kernels in one .cl/.aocx file, where each kernel uses about the same amount of local memory, will AOCL compiler able to generate a single memory block and share it between the kernels? or will it just instantiate a different local memory for each kernels? The kernels that I have will not be running concurrently. The previous kernel must finish before the next can start; and I would like to allocate as much local memory as possible. I am just asking because the AOCL release note mentioned that "num_share_resources", "max_share_resources" and "max_unroll_loops" are deprecated and no work around was given.

    Also, I am wondering if there is a way to tell the AOCL compiler explicitly that I won't run the different kernels contained in the same file concurrently so that it could optimize the hardware better, or does that not make any difference anyway?

    Regards,

    Ryan

    --- Quote Start ---

    It is the latter; the programming guide recommends that "the entire kernel should have 4 or fewer different accesses to local memory.

    Basically, each load/store instruction in the kernel becomes a client (i.e. master) for local memory. Because local memory has at most 4-ports, if you have 4 load/store instructions, each port will be connected to a single load/store so that load/store instructions will not compete with each other. This guarantees the most efficient hardware.

    If you have 3 or fewer store instructions and many loads (loads+stores > 4), the compiler may choose to replicate the local memory. This also gives fast accesses at the local memory at the expense of RAMs.

    Further, the compiler performs some optimizations to partition the local memory based on the access patterns. Hence, even if there are more than 4 store instructions, you may still get efficient hardware. However, this depends on the complexity of your access patterns and may not always be possible.

    --- Quote End ---

  • Altera_Forum's avatar
    Altera_Forum
    Icon for Honored Contributor rankHonored Contributor
    11 years ago

    Also, does it matter if I use the local memory in 2d (local_mem[i][j]) or flatten it to 1d (local_mem[i*block_size + j]) ?

  • Altera_Forum's avatar
    Altera_Forum
    Icon for Honored Contributor rankHonored Contributor
    11 years ago

    There is currently no way to specify that kernels in a source file are will be launched/run non-concurrently. The compiler always assumes that they may run simultaneously, so there is no resource sharing (either local memory or other resources) across distinct kernels. If you want to share resources, then you can currently do this manually - define a single kernel with a local memory system, and then call sub-functions from that kernel that define the functionality of the independent kernels. You can pass a local memory pointer to those "kernel" sub-functions.

  • Altera_Forum's avatar
    Altera_Forum
    Icon for Honored Contributor rankHonored Contributor
    11 years ago

    Both 2d and 1d array accesses should produce similar results, although in some special (complex) cases the compiler might have an easier time analyzing/optimizing the 2d notation. Since 2d is probably your preferred implementation, I don't think there is any reason to flatten the accesses.

  • Altera_Forum's avatar
    Altera_Forum
    Icon for Honored Contributor rankHonored Contributor
    11 years ago

    Thank you! I am just wondering if I call a "sub" function from my "main" kernel multiple times (ie function call is under a "for" loop) would it cause higher overhead, or cause the compiler to use more hardware? Will this way (use one kernel to call sub functions) of programming in general cause the higher hardware utilization or degrade performance compare to having multiple smaller kernels?

    --- Quote Start ---

    There is currently no way to specify that kernels in a source file are will be launched/run non-concurrently. The compiler always assumes that they may run simultaneously, so there is no resource sharing (either local memory or other resources) across distinct kernels. If you want to share resources, then you can currently do this manually - define a single kernel with a local memory system, and then call sub-functions from that kernel that define the functionality of the independent kernels. You can pass a local memory pointer to those "kernel" sub-functions.

    --- Quote End ---

  • Altera_Forum's avatar
    Altera_Forum
    Icon for Honored Contributor rankHonored Contributor
    11 years ago

    Thank you! BTW: Is there a difference in latency/throughput between accessing a 2D local memory row by row or accessing it column by column?

  • Altera_Forum's avatar
    Altera_Forum
    Icon for Honored Contributor rankHonored Contributor
    11 years ago

    --- Quote Start ---

    Thank you! I am just wondering if I call a "sub" function from my "main" kernel multiple times (ie function call is under a "for" loop) would it cause higher overhead, or cause the compiler to use more hardware?

    --- Quote End ---

    The compiler inlines functions into hardware instead of "calling" them, so the answer depends on whether the loop gets automatically unrolled. Auto-unrolling can occur if the loop body is relatively small, and if the loop has a small fixed trip count. If unrolled, the function that you're calling will be replicated multiple times which uses additional hardware. Similarly, if you call the function multiple times from within a kernel, then you will incur the overhead of the function being instantiated multiple times in the hardware. The compiler does this because replicating the hardware provides higher throughput when you have many workitems - different sets of work items can use different instances of the function in parallel.

    --- Quote Start ---

    Will this way (use one kernel to call sub functions) of programming in general cause the higher hardware utilization or degrade performance compare to having multiple smaller kernels?

    --- Quote End ---

    Yes, it will most likely degrade the performance of all of the kernels. Local memory systems are optimized for the kernel that they connect to. When you fuse the functionality of multiple kernels, each of which likely has a different type of memory access pattern, you will end up with a complex local memory system that isn't optimized for any one of the kernels. I would expect to see more stalls/access conflicts on the memory, and a higher hardware cost.

    The single fused kernel would also be much more complex, which may prevent some compiler optimizations (especially memory) that could be performed on each of the smaller, simpler kernels.

    Your previous question asked how to effectively share a local memory across kernels. That's the definition of a global memory, so if at all possible, I would suggest re-working your multiple kernels to either: (1) share global memory instead of local; or (2) find a reasonable balance between local memory sizes such that each kernel can have its own local mem. If you care about throughput/performance and not just fitting multiple kernels each requiring massive local memory onto the chip (at the cost of performance), then fusing kernels is probably not the best way to go.