Steve Whealton

Squeezing Numbers

Often it will happen that numbers are fed in, a formula is used on them, and a final number is produced. Now, it's time to put that number back somewhere; to determine the color of a pixel, or the pitch or loudness of a musical tone. Normally, the computer will accept only a limited range of numbers for its pixel colors or its pitches. Typically also, formulas spew out numbers over a far wider range.

Banding

Suppose you are using a formula that gives you numbers ranging from -10,000 to +10,000, but your computer can only handle 16 colors. Further, your computer (it's software) has named those 16 available colors by means of the numbers from 0 up to 15. Your task, then, is to take that huge and unwieldy gamut of given numbers (-10,000 to +10,000) and to squeeze it down into the required range (0 to 15).

The simplest and most straightforward method is simply to consider the range of possible incoming numbers, and to create equal-sized bands out of them. The incoming range here goes from -10,000 to +10,000, a total distance of 20,000. Dividing 16 into this range, one comes up with 16 bands, each one of size 1,250. These 16 bands must now be properly stretched out within the original range.

From -10,000 up to -8,750 will correspond with 0; from -8,749 up to -7,500 will correspond to 1, and so on. Numbers from 8,750 up to 10,000 will correspond with 15.

Simply programmed, easily understood; banding is also boring. My formulas are typically rich in detail. By squeezing them via banding, I lose all subtlety of image.

For this reason, I use banding rarely. Sometimes, I might use it in tandem with other tools. On occasion, I might have reason to want all detail to be lost.

Modulo

Even easier to program than banding is the modular arithmetic approach. Even better, it almost always gives subtly detailed images.

Let us look again at the squeezing problem considered above. The incoming range is from -10,000 to 10,000, while the outgoing range is from 0 to 15.

One approach would be first to take the absolute value of the incoming number and then to divide the result by 16.

Another approach would be to add 10,000 (to put all numbers out of the negative range) and then to divide that result by 16.

In either case, the next step is to look not at the quotient, but rather at the remainder.

There are other ways of squeezing numbers, but this particular use of modular arithmetic is so easy (a = b mod 16) and so often so satisfying, that I use it right by itself for well over half of my squeezings.