koobas.hobune.stream
Unity/C# Division problem
Given this code:
print (dP.transform.localPosition.x + " " + TileManager.tileSize);
print (1.3f/1.3f);
print (dP.transform.localPosition.x/TileManager.tileSize);
I get this as output:
1.3 1.3
1
0.9999998
The kicker is that all of the positions are exactly set from the editor as 1.3. Datatypes are all float. Viewing values in inspector and debugger shows no difference during runtime, the value of dP.transform.localPosition.x is 1.3!
I know there are floating point issues when doing division but I am doing the exact same operation both ways. Does it have to do with using a constant value vs a stored value?
Yea I am just wondering why 1.3f/1.3f = 1 and referencing a variable storing 1.3f and dividing it by another variable storing 1.3f gives a different answer.
The expression "1.3f/1.3f" is resolved as a constant at compile time and will be replaced with "1f". Anyways the string conversion might also round the value
Likely what's happening here is that the compiler sees you're doing math with two constants and so calculates the result at compile-time instead of waiting till run-time. This compile-time math "engine" is apparently able to take a shortcut when the two operands are the same, but the run-time version of division is not.
The runtime calculation is producing a result that is not unusual for floating point numbers. You could use Math.Round to get the 1.0 that you want.
This will give you the exact same result:
float f1 = 1.2999998f;
float f2 = 1.3000001f;
Debug.Log (f1 +" "+ f2);
Debug.Log (1.3f / 1.3f);
Debug.Log (f1 / f2);
The problem is that the string conversion might round the last digit away. So your values actually aren't equal. That's absolutely normal for floating point values. Also .NET / Mono is even allowed to cast floats to doubles, executing an operation and cast back to float. This only happens in rare cases but might result in a slightly different result.
If you add this line:
Debug.Log (f1.ToString("G20") +" / "+ f2.ToString("G20"));
you will see that f1 and f2 aren't 1.3 nor 1.2999998f nor 1.3000001f as they can't be represented as float. A float is stored as binary number, not as decimal.
ps: Even this line:
Debug.Log("Test: " + 1.3f.ToString("G20"));
will print this:
"Test: 1.29999995"
See Single precision floating point format for more information.
