Optimization for complex algorithms

Of course. The DSPs in Stratix V only natively support integer operations. For floating-point multiplication, large parts of the shifting and rounding will have to be performed outside of the DSP which consumes a significant amount of logic and memory. Note that only floating-point multiplication will use DSPs on Stratix V (alongside with logic and memory), while every other floating-point operation (including addition) will only use logic and memory. Adding --fp-relaxed might or might not make any difference, but on Stratix V, adding --fpc must make a very noticeable difference. The difference will also reflect in the report; if you are not seeing it in the report, it means the switch has not been applied correctly. Assuming that you are using a makefile, have you made sure the switches are being correctly applied to the aoc command?

Considering the fact that you are using Stratix V, the high memory and logic utilization you are seeing is because of using floating-point operations and even with --fpc, you might not be able to fit the design. Switching to fixed-point as outlined in Altera's documents could also help.

Reducing number of buffers, and also number of accesses to each buffer, can help reduce the memory utilization. Assuming that you are using an NDRange implementation, the compiler report will give detailed info about which buffers are implemented using memory blocks, and how many accesses exist to each buffer, and how many times the buffer has been replicated to support these accesses. Though I am not sure if this info still exists in latest versions of the compiler; I haven't used NDRange in a long time.