Output amplitute from an NCO

sinc distortion is caused by the hold effect of DAC(it doesn't affect the DAC value conversion itself, i.e. the ladder of values will always follow the conversion rules) but the hold effect is very much like any averaging low pass filter(analogue-wise).

The amplitude droop equation is this:

a = A * sine(x)/x where x = pi * f/Fs, A is amplitude at f without distortion

the inverse to be applied is x/sine(x). This is the frequency response required from the correction filter. You then apply ifft in Matlab to get coefficients.

Another remedy is to upsample so high that your signal is always far away from main shoulder of distortion.

make a search for inverse sinc and there is alot around.