If you are new to programming, then this lesson may contain some surprising information. But even if you are used to writing programs, computers can do unexpected things with numbers. The purpose of this lesson is to shed some light on some of the mysteries and quirks you can encounter.
These mysteries exist independently of the programming language, though one programming language may be better at isolating you from them than another. The problem is that computers can not deal with the numbers we are used and in the way we are used to.
For instance (*):
# Compute 1 million times 1 million % puts [expr {1000000*1000000}] -727379968To most people's surprise, the result is negative! Instead of 1000000000000, a negative number is returned.
Important note: I used Tcl 8.4.1 for all examples. In Tcl 8.5 the results will hopefully be more intuitive, as a result of adding so-called big integers. Nevertheless, the general theme remains the same.
Now consider the following example, it is almost the same, with the exception of a decimal dot:
# Compute 1 million times 1 million % puts [expr {1000000*1000000.}] 1e+012The reason is simple, well if you know more about the background of computer arithmetic:
Numbers outside that range can not be dealt with that easily. This is also true of the results of a computation (or the intermediate results of a computation, even if the final result does fit).
The typical range for floating-point numbers is roughly: -1.0e+300 to -1.0e-300, 0.0 and 1.0e-300 to 1.0e+300. Floating-point numbers have a limited precision: usually about 12 decimals.
What this means in practical terms, is that a floating-point number can be as a 1 followed by 300 zeros or as small as 0.0000...1 (where "..." stands for 295 zeros).
Because the range is so much bigger, in the second example the result falls within the limits and we get the answer we expected.
Tcl uses a simple but efficient strategy to decide what kind of numbers to use for the computations:
Floating-point computations are quite complex, and the current (IEEE) standard prescribes what should happen in minute detail. One such detail is that results outside the proper ranges are reported. Tcl catches these and displays a warning:
# Compute 1.0e+300/1.0-300 % puts [expr {1.0e300/1.0e-300}] floating-point value too large to represent
Now some of the mysteries you can find yourself involved in. Run the following scripts:
# # Division # puts "1/2 is [expr {1/2}]" puts "-1/2 is [expr {-1/2}]" puts "1/2 is [expr {1./2}]" puts "1/3 is [expr {1./3}]" puts "1/3 is [expr {double(1)/3}]"
The first two computations have the surprising result: 0 and -1. That is because the result is an integer number and the mathematically exact results 1/2 and -1/2 are not.
If you interested in the details of how Tcl works, the outcome q is determined as follows:
a = q * b + r 0 <= |r| < |b| r has the same sign as q
Here are some examples with floating-point numbers:
set tcl_precision 17 ;# One of Tcl's few magic variables: ;# Show all decimals needed to exactly ;# reproduce a particular number puts "1/2 is [expr {1./2}]" puts "1/3 is [expr {1./3}]" set a [expr {1.0/3.0}] puts "3*(1/3) is [expr {3.0*$a}]" set b [expr {10.0/3.0}] puts "3*(10/3) is [expr {3.0*$b}]" set c [expr {10.0/3.0}] set d [expr {2.0/3.0}] puts "(10.0/3.0) / (2.0/3.0) is [expr {$c/$d}]" set e [expr {1.0/10.0}] puts "1.2 / 0.1 is [expr {1.2/$e}]"
While many of the above computations give the result you would expect, note however the last decimals, the last two do not give exactly 5 and 12! This is because computers can only deal with numbers with a limited precision: floating-point numbers are not our mathematical real numbers.
Somewhat unexpectedly, 1/10 also gives problems. 1.2/0.1 results in 11.999999999999998, not 12. That is an example of a very nasty aspect of most computers and programming languages today: they do not work with ordinary decimal fractions, but with binary fractions. So, 0.5 can be represented exactly, but 0.1 can not.
The fact that floating-point numbers are not ordinary decimal or real numbers and the actual way computers deal with floating-point numbers, has a number of consequences:
# # The wrong way # set number [expr {int(1.2/0.1)}] ;# Force an integer - ;# accidentally number = 11 for { set i 0 } { $i <= $number } { incr i } { set x [expr {$i*0.1}] ... create label $x } # # A right way - note the limit # set x 0.0 set delta 0.1 while { $x < 1.2+0.5*$delta } { ... create label $x set x [expr {$x + $delta}] }
# # Two different estimates of "pi" - on Windows 98 # set pi1 [expr {4.0*atan(1.0)}] set pi2 [expr {6.0*asin(0.5)}] puts [expr {$pi1-$pi2}] -4.4408920985006262e-016