Floating point comparisons…

I was talking with a colleague recently about floating point equality comparison. A double was being serialized, de-serialized and then compared and the loss of precision through the process resulted in the before and after versions being not quite equal.

His initial attempts tried to reconstruct the exact value (from a textual serialization format) after de-serialization by carefully saving all available precision. This failed and I think it was the wrong approach in any case. It is nearly impossible to be sure that every last bit of a fractional floating point value is preserved without  preserving the original bit pattern (and this defeats the purpose of saving a human editable form like JSON or XML).

The first discussion centered around floating point epsilon values (least significant bit of the type in use). In this particular case though this wasn’t really the issue. He didn’t need the full precision of the type, his application just needed to test that the values were close enough.

I think the only viable way to implement this sort of comparison robustly involved the site of the compare. Once you define the required degree of accuracy (generally you’ll be discussion base 10 digits after the decimal point) you need to check that the two values are within 1/2 of the next digit down (below and above).

This allows loss of precision due to manipulations (may be serialization, may be computation) while preserving the important parts of the comparison.

Figure out how much precision you need. You’ll wind up with

fuzz = 10^(-(digits + 1) * (1/2)

as the ‘fuzziness’ in the comparison.

If the first value is less than the second value plus fuzz and greater than  the second value minus fuzz then you’ve got a good enough match. In any other case you don’t have a match.

While I wouldn’t recommend using floating point values for this sort of thing  in the first place, if you’re stuck with one in your system, this approach should preserve your sanity and keep things from getting too weird…


One thought on “Floating point comparisons…”

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.