Quantization

In digital signal processing, quantization is the rounding of values to a predefined set of values. The original values can be from a continuous set, such as R\mathbb{R}, or it can be from a discrete set, such as N\mathbb{N}. In either case, the resulting values will be a smaller set.

For example, we can take the values in a range from 0 - 10 and round every input value to the closest multiple of 2.5. If we were given inputs (0.3,0.7,1.2,1.9,5.7,9.9)(0.3, 0.7, 1.2, 1.9, 5.7, 9.9) we'd get outputs (0,0,0,2.5,5,10)(0, 0, 0, 2.5, 5, 10). This is a very rough quantization as numbers are being rounded by a significant amount.

All uses of quantization result in some amount of error. The act of quantization makes it impossible to fully recreate the original input. A larger output set (in terms of granularity, not range) results in being able to reconstruct the original input signal more accurately.

Quantization can be thought of sampling but on the y-axis, as opposed to the x-axis.

As another example, here's a case of quantizing a sine wave such that it has 4 potential values: 0, 1, 2, 3. It also has a sampling rate of 8 samples per cycle. It has a bit depth of 2, as 2 bits can be used to represent all potential values. This would sound terrible.

Pasted image 20230115171226 By Hyacinth - Own work, CC BY-SA 3.0, https://commons.wikimedia.org/w/index.php?curid=23867344.