- Home /
Question by
Mun-Yeong-Seok · Sep 26, 2017 at 08:50 PM ·
calculationfloatingpoint
(long) type conversion is a bit inaccurate.
print((long)(0.0f * 10.0f));
print((long)(0.1f * 10.0f));
print((long)(0.2f * 10.0f));
print((long)(0.3f * 10.0f));
print((long)(0.4f * 10.0f));
print((long)(0.5f * 10.0f));
print((long)(0.6f * 10.0f));
print((long)(0.7f * 10.0f));
print((long)(0.8f * 10.0f));
print((long)(0.9f * 10.0f));
print("---------");
print((long)(1.0f * 10.0f));
print((long)(1.1f * 10.0f));
print((long)(1.2f * 10.0f));
print((long)(1.3f * 10.0f));
print((long)(1.4f * 10.0f));
print((long)(1.5f * 10.0f));
print((long)(1.6f * 10.0f));
print((long)(1.7f * 10.0f));
print((long)(1.8f * 10.0f));
print((long)(1.9f * 10.0f));
print((long)(2.0f * 10.0f));
Why are these results coming out? Do I miss something? I want to increase the number of floating-point and convert it to a (long) type, but the value is incorrect.
123.png
(46.6 kB)
Comment
Best Answer
Answer by Kishotta · Sep 26, 2017 at 09:25 PM
This is a limitation of how computers handle floating point numbers. Instead of casting your result directly to a long, use:
(long)Mathf.RoundToInt (0.1f * 10f)
That should round the result to an int properly before casting to long.
How large are the numbers you're needing to store?
Thanks! The parameter type of ReportScoreTo$$anonymous$$erboard() is long, so I cast it to long type.