Unity Mathf Float Minus Integer Rounding Error

The behavior you're seeing is inherent to floating-point representations (generally speaking), and is to be expected. You can't really prevent it per se, but there are ways to accommodate it (how to accommodate it depends largely on the context).

[Edit: I haven't provided an explanation of why this happens, as that information is widely available already. Here though is a link to get you started if you want to investigate it further; the paper is pretty dense, but it's more or less a standard reference on this subject. If you search online for 'floating-point error' though, you can probably find some easier-to-follow explanations.]

Edit: In response to the comment below...

It's inherent because floating-point representations use a finite number of bits to represent an infinite number of real numbers. Therefore, only a finite subset of the real numbers can actually be represented. One consequence of this (among many) is that many numbers cannot be exactly represented; instead, the nearest 'machine number' must be selected. That's one reason that floating-point values don't always come out exactly as you'd expect them to.

(The full story behind the floating-point representations most commonly used today is pretty involved; again, check the paper I linked for more info.)

As for accommodating this inherent imprecision, there are various solutions, but a common one is to introduce an epsilon or tolerance value. That is, rather than asking if x = y, you ask whether x equals y within some specified tolerance.