Forum Discussion
Occasional ContributorAI Suite - Is layout transformation useful in more scenarios?
Frequent ContributorHi Mads
Apologies in delay in getting back, took me sometime to go through the layout transformation section as well.
“Current input and filter go into the same feature for the PE this cycle”
Mostly correct for inference. In a PE cycle, the engine combines activations (input features) with filter weights. Layout transform only prepares the input; filters come from the compiled graph, not from the layout-transform step.
Is it possible to process more channels, so it goes up the timeline of input and also folds that into the current input?
Folding does let the PE see more channels in one go. Those extra channels are nearby pixels in space (height/width, and sometimes depth as well), based on the first convolution’s stride. They are packed into the channel axis of the current tensor and not as later samples along an input stream or time axis. So it is not “look ahead in time and merge that into now.” It is “reshape the spatial neighborhood into channels so this PE cycle is fuller.”
https://docs.altera.com/r/docs/863373/2025.3/fpga-ai-suite-handbook/folding-input
"Is this only useful when doing image stuff or can it also be utilized when doing simple linear stuff?"
Short answer is both image stuff and linear stuff can use layout transform but folder is mainly more useful for images / first-convolution jobs.
Some further details:
- Layout transform (reordering and packing the input into the format the PE array expects, including type conversion to FP16 when needed) is useful whenever the input is not already in that format. That applies to image tensors and to simpler linear/FC feature vectors.
- Folding is different. It packs spatial neighbors (height/width, based on the first convolution’s stride) into the channel dimension so a shallow first layer (for example RGB) uses the PE array more efficiently. A simple linear layer is typically just a feature vector with little or no spatial H×W structure, so folding usually provides little benefit.
In short for simple linear networks: use layout transform if you need the correct input packing/format. Do not expect folding to be the main performance lever the way it is for shallow first-layer convolutions on images
https://docs.altera.com/r/docs/863373/2025.3/fpga-ai-suite-handbook/folding-input
https://docs.altera.com/r/docs/863373/2025.3/fpga-ai-suite-handbook/transforming-input-data-layout
"In the compiler it is possible to optimize for area and resources, it is not possible to optimize for throughput which is the most important. For transformation the only parameter is --folding which must be set manually. When should I set the folding parameter? when can I utilize this to improve my design?"
I assumed the --folding you mentioned is the --folding-option (0-3) where
- 0 - no folding (useful for debug, often lower first-layer efficiency)
- 1 - folding by host runtime or the IP layout-transform module
- 2 - host folding, plus extra folding in the IP convolution engine
- 3 - all folding in the IP convolution engine (default)
Use or keep folding option (default 3, or 1 if you want the layout-transform path involved) when:
- the first layer is a convolution
- input channels are shallow (for example 1, or RGB = 3)
- architecture CVEC is large
- first-convolution stride > 1×1
This is the case where folding usually improves the first-layer PE utilization and can help with throughput.
Turn folding off or use lightweight layout transform when:
- you are debugging
- the network is mostly linear / FC (little or no spatial H×W to fold)
- first-convolution stride is 1×1 (folding often adds little)
- the full layout transform costs too much ALM/DSP area
reference link where i get the info, perhaps more details in the link (actually same document just different section)
https://docs.altera.com/r/docs/863373/2025.3/fpga-ai-suite-handbook/folding-input
https://docs.altera.com/r/docs/863373/2025.3/fpga-ai-suite-handbook/lightweight-layout-transform
Thanks
Regards
Kian
