7.2 Integer Truncation

There are four commands for truncating a real number to an integer, differing mainly in their treatment of negative numbers. All of these commands have the property that if the argument is an integer, the result is the same integer. An integer-valued floating-point argument is converted to integer form.

If you press H (calc-hyperbolic) first, the result will be expressed as an integer-valued floating-point number.

The F (calc-floor) [floor or ffloor] command truncates a real number to the next lower integer, i.e., toward minus infinity. Thus 3.6 F produces 3, but _3.6 F produces -4.

The I F (calc-ceiling) [ceil or fceil] command truncates toward positive infinity. Thus 3.6 I F produces 4, and _3.6 I F produces -3.

The R (calc-round) [round or fround] command rounds to the nearest integer. When the fractional part is .5 exactly, this command rounds away from zero. (All other rounding in the Calculator uses this convention as well.) Thus 3.5 R produces 4 but 3.4 R produces 3; _3.5 R produces -4.

The I R (calc-trunc) [trunc or ftrunc] command truncates toward zero. In other words, it “chops off” everything after the decimal point. Thus 3.6 I R produces 3 and _3.6 I R produces -3.

These functions may not be applied meaningfully to error forms, but they do work for intervals. As a convenience, applying floor to a modulo form floors the value part of the form. Applied to a vector, these functions operate on all elements of the vector one by one. Applied to a date form, they operate on the internal numerical representation of dates, converting a date/time form into a pure date.

There are two more rounding functions which can only be entered in algebraic notation. The roundu function is like round except that it rounds up, toward plus infinity, when the fractional part is .5. This distinction matters only for negative arguments. Also, rounde rounds to an even number in the case of a tie, rounding up or down as necessary. For example, ‘rounde(3.5)’ and ‘rounde(4.5)’ both return 4, but ‘rounde(5.5)’ returns 6. The advantage of round-to-even is that the net error due to rounding after a long calculation tends to cancel out to zero. An important subtle point here is that the number being fed to rounde will already have been rounded to the current precision before rounde begins. For example, ‘rounde(2.500001)’ with a current precision of 6 will incorrectly, or at least surprisingly, yield 2 because the argument will first have been rounded down to ‘2.5’ (which rounde sees as an exact tie between 2 and 3).

Each of these functions, when written in algebraic formulas, allows a second argument which specifies the number of digits after the decimal point to keep. For example, ‘round(123.4567, 2)’ will produce the answer 123.46, and ‘round(123.4567, -1)’ will produce 120 (i.e., the cutoff is one digit to the left of the decimal point). A second argument of zero is equivalent to no second argument at all.

To compute the fractional part of a number (i.e., the amount which, when added to ‘floor(n)’, will produce n) just take n modulo 1 using the % command.

Note also the \ (integer quotient), f I (integer logarithm), and f Q (integer square root) commands, which are analogous to /, B, and Q, respectively, except that they take integer arguments and return the result rounded down to an integer.