Hi,
Thanks for your clarification that the Matlab result matches the FPGA simulation. Based on this, it seems like the observation that you are seeing is expected behavior of the FFT under your specific configuration. This seems to be trending towards FFT theoretical behavior and I would like to get further information from you to see if find any clue for your inquiry:
1. Would you mind share with me the FFT .ip file generated by Quartus so that I can have a better understanding of your configuration?
2. What is the Quartus version of FPGA device that you are using?
3. Would you mind to share with me the Matlab simulation or the FPGA simulation so that I can have a better understanding on the output precision observation that you are referring to?
4. Please feel free to let me know the Matlab version as well as the Modelsim version that you are using for simulation.
Please let me know if there is any concern. Thank you.
Best regards,
Chee Pin