Veerasundaravel's Ruby on Rails Weblog

March 30, 2010

Ruby Float comparison issues

Filed under: Ruby — Tags: , , , , , , — Veerasundaravel @ 5:43 am

When I tried to compare a floating value expression, I got strange output like below

>> (1.2 – 1.0) == 0.2
>> false

Floating point numbers represent an extremely wide range of values – much wider than their integer counterparts. This is handled through an exponent and mantissa. For this ability, they trade off precision. Think about the case of adding a large floating point number to a small floating point number:

So assume if you are going to store money values as float means, you will loss the precision and it will give you false comparision. On this case use smallest denomination such as cents, and convert the cents to dollors in front end or use a custom Money column type.

We can use Decimal arithmetic which is also useful for general calculation, because it provides the correct comparision for these type of float numbers, where floating point arithmetic often introduces subtle errors because of the conversion between base 10 and base 2.

So, if you need to compare Floats and you are not receiving the expected response, you can use the BigDecimal class instead:

(BigDecimal.new(“1.2”) – BigDecimal(“1.0”)) == BigDecimal(“0.2”)

Advertisements

1 Comment »

  1. Thanks a lot :)

    Comment by SteveRob — August 5, 2012 @ 11:31 am


RSS feed for comments on this post. TrackBack URI

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: