(float)100 * 0.01f is 0, why??
int a = 100;
int b = (int) ((float)a * 0.01f);
debug.log(b);
after run, b is 0,
I cannot understand that result. I expected 1, but result was 0,
Why result is 0 ??
Are you talking about multiplication like in your headline or division like in your question?
And why do you expect one tenth of 100 to be 1?
Answer by Jessespike · Nov 13, 2015 at 04:19 PM
Well, first off: 100 / 0.1 results in 1000. And 100 x 0.1 results in 10. Not sure why you're expecting it to be 1. Did you write the right numbers in the question?
And secondly: Variable b is an int which can only store whole number values (no decimal places). So if you were to 0.1 / 100 and get 0.001, storing that result as an int will make it round to 0.
Sorry, $$anonymous$$y mistake. I edited my question. in unity editor, int b = 100 0.01f, value of b was 0, but iPhone, result of 100 0.01f is 1. I could not understand it.
Answer by joemane22 · Nov 16, 2015 at 06:48 AM
Sounds like you might be running into a floating point rounding error, in floating point it might be coming out as 0.9(recursively) which is still a zero as far as integers are concerned. Using double instead might fix it or calling
int b = Mathf.RoundToInt(a * 0.01f);
But be advised that will cause 0.5f to come out as 1 also.
Your answer
Follow this Question
Related Questions
Floating point imprecision at a mere ~5km from origin 0 Answers
Oculus Quest Floating Point Errors and Large Scale Worlds 0 Answers
Non physics determinism of floating point math across platforms 1 Answer
Floating point error message in shader 0 Answers
Weird lines, occur more often on android device, need solution to get rid of the lines. 1 Answer