Forum Discussion

Altera_Forum's avatar
Altera_Forum
Icon for Honored Contributor rankHonored Contributor
11 years ago

sqrtf custom instruction gives sometimes absolutely wrong results

I use Quartus 13.1.0.182 web edition with a Cyclone III FPGA The project is compiled with the Floating point custom instruction 2.0 from Qsys. My code calculates the square root and some time...
Altera_Forum's avatar
Altera_Forum
Icon for Honored Contributor rankHonored Contributor
11 years ago

Looks that way, probably failing to increment the exponent when the mantissa is rounded up.

FWIW you'll probably get better precision from calculating one of sqrt((1+sin)*(1-sin)) or sqrt(0.5 - sin*sin + 0.5).