Floating point error in Python

Python, a widely used programming language, excels in numerical computing tasks, yet it is not immune to the challenges posed by floating-point arithmetic. Floating-point numbers in Python are approximations of real numbers, leading to rounding errors, loss of precision, and cancellations that can throw off calculations. We can spot these errors by looking for strange results and using tools numpy.finfo to monitor precision. With some caution and clever tricks, we can keep these errors in check and ensure our Python calculations are reliable. In this article, we will explore the intricacies of floating-point errors in Python.

What are Floating Point Numbers?

Floating-point numbers are an efficient way to represent real numbers in computers. They consist of three parts:

• Significant: The actual digits representing the number (e.g., 3.14159)
• Exponent: Tells how many places to shift the significand to the left or right (e.g., -2 in 3.14159 x 10^-2)
• Base: Typically 2 for computers, determining how numbers are represented internally

Why do floating-point errors occur?

Floating-point errors arise because computers store real numbers using a finite number of bits, leading to approximations and potential inaccuracies. Floating-point numbers have intrinsic limitations:

• Finite precision: Only a limited number of digits can be stored in the significand, leading to rounding errors when representing exact decimals.
• Loss of precision: Operations like addition or subtraction can further reduce precision, compounding the effects of rounding.
• Underflow/Overflow: Extremely small or large numbers can fall outside the representable range, leading to underflow (becomes zero) or overflow (becomes infinity).

Types of Floating-Point Errors

a) Rounding errors: The most common, occurring when an exact decimal has to be approximated to fit the limited precision of a float.

b) Loss of precision: Subsequent operations can gradually accumulate rounding errors, leading to significant inaccuracies in the final result.

c) Catastrophic cancellation: When subtracting nearly equal numbers with opposite signs, their significant digits cancel out, leaving a small and inaccurate result.

d) Overflow/Underflow: These occur when calculations exceed the representable range of float values, leading to inaccurate or meaningless results.

Detecting Floating-Point Errors

1. Observing unexpected results: Comparing calculated values to expected outcomes or visualizing data can reveal inconsistencies often caused by errors.
2. Using libraries like numpy.finfo: Libraries like numpy provide tools like finfo to check the precision and limitations of different float data types.

Python Floating Point Error

Here we will discuss different types of examples that illustrate floating-point errors in Python:

Loss of Precision in Decimal to Binary Conversion

In this example, the decimal number 0.1 is converted to binary. Due to the infinite binary expansion of 0.1, only a finite number of bits are used, leading to a loss of precision.

Output:

Decimal: 0.1
Binary: 0.100000000000000005551115123126

Rounding Errors

Here, the result of the addition of 1/3 three times is expected to be 1.0. However, due to rounding errors in representing 1/3, the sum may not be exactly 1.0.

Output:

Expected Result: 1.0
Actual Result: 1.0

Accumulative Errors in Iterative Calculations

This example demonstrates how accumulative errors can occur in iterative calculations. Adding 0.1 ten times may not yield an exact result of 1.0 due to floating-point precision limitations.

Output:

Expected Result: 1.0
Actual Result: 0.9999999999999999

Comparison Issues

In this case, comparing the sum of 0.1 and 0.2 to 0.3 may not yield the expected True result due to the inherent imprecision of floating-point numbers.

Output:

a: 0.30000000000000004
b: 0.3
Equal: False

Unexpected Results in Calculations

Here, the subtraction of 1e16 from the sum (1e16 + 1) is expected to yield 1, but due to floating-point errors, the result may not be exactly 1.

Output:

Expected Result: 1
Actual Result: 0.0

Understanding Floating-Point Precision

Here we will understand the floating point precision: The 1.2 - 1.0 Anomaly in Python-

Representation Challenges

As it is known that 1.2 - 1.0 = 0.2. But when you try to do the same in Python you will surprised by the results:

>>> 1.2 - 1.0

Output:

0.199999999999999996

This can be considered a bug in Python, but it is not. This has little to do with Python and much more to do with how the underlying platform handles floating-point numbers. It's a normal case encountered when handling floating-point numbers internally in a system. It’s a problem caused when the internal representation of floating-point numbers, which uses a fixed number of binary digits to represent a decimal number. It is difficult to represent some decimal numbers in binary, so in many cases, it leads to small roundoff errors. We know similar cases in decimal math, many results can’t be represented with a fixed number of decimal digits, for Example

10 / 3 = 3.33333333.......

In this case, taking 1.2 as an example, the representation of 0.2 in binary is 0.00110011001100110011001100...... and so on. It is difficult to store this infinite decimal number internally. Normally a float object’s value is stored in binary floating-point with a fixed precision (typically 53 bits). So we represent 1.2 internally as,

1.0011001100110011001100110011001100110011001100110011 

Which is exactly equal to :

1.1999999999999999555910790149937383830547332763671875

Handling Floating Point Error

Here we will discuss different example on how to handle floating point errors in Python:

Rounding to a Specific Decimal Place

By rounding the result to a specific decimal place (e.g., 2), you can mitigate the impact of small floating-point errors.

Output:

Original Result: 0.19999999999999996
Rounded Result: 0.2

Using Decimal Class for High Precision

The decimal module provides the Decimal class, allowing for higher precision arithmetic. Setting the precision with getcontext().prec can help in managing precision for specific calculations

Output:

High Precision Result: 0.2

Using Fractions for Exact Representations

The fractions module allows working with exact fractional representations, avoiding floating-point errors.

Output:

Exact Fractional Result: 1/5

Handling Intermediate Results with Decimal

Use the Decimal class for intermediate calculations to minimize cumulative errors before converting back to float.

Output:

Intermediate Result: 0.2
Final Result: 0.2

Conclusion

Still, you thinking why python is not solving this issue, actually it has nothing to do with python. It happens because it is the way the underlying c platform handles floating-point numbers and ultimately with the inaccuracy, we'll always have been writing down numbers as a string of fixed number of digits. Note that this is in the very nature of binary floating-point: this is not a bug either in Python or C, and it is not a bug in your code either. You’ll see the same kind of behaviors in all languages that support our hardware's floating-point arithmetic although some languages may not display the difference by default, or in all output modes). We have to consider this behavior when we do care about math problems with needs exact precisions or using it inside conditional statements. Check floating point section in python documentation for more such behaviours.