Forum Discussion

Altera_Forum's avatar
Altera_Forum
Icon for Honored Contributor rankHonored Contributor
14 years ago

ALTFP_INV_SQRT problem

I have instanced ALTFP_INV_SQRT (y = 1/sqrt(y)) mega function in Quartus 10. I’m using single precision. When I calculate X”3F800001" (almost 1) the result is X”3F000000” (0.5). Calculating X” 3F800000" and X” 3F800002" is OK as shown below.

Simulations:

X"3F800000" => X"3F800000" or

1,0000000000000000 => 1,0000000000000000

X"3F800001" => X"3F000000" or

1,0000001192092896 => 0,5000000000000000 -ERROR

X"3F800002" => X"3F7FFFFE" or

1,0000002384185791 => 0,9999998807907104