Quantization (digital signal processing)
In digital signal processing, quantization is the rounding of values to a predefined set of values. The original values can be from a continuous set, such as , or it can be from a discrete set, such as . In either case, the resulting values will be a smaller set.
For example, we can take the values in a range from 0 - 10 and round every input value to the closest multiple of 2.5. If we were given inputs we’d get outputs . This is a very rough quantization as numbers are being rounded by a significant amount.
All uses of quantization result in some amount of error. The act of quantization makes it impossible to fully recreate the original input. A larger output set (in terms of granularity, not range) results in being able to reconstruct the original input signal more accurately.
Quantization can be thought of sampling but on the y-axis, as opposed to the x-axis.
As another example, here’s a case of quantizing a sine wave such that it has 4 potential values: 0, 1, 2, 3. It also has a sampling rate of 8 samples per cycle. It has a bit depth of 2, as 2 bits can be used to represent all potential values. This would sound terrible.
By Hyacinth - Own work, CC BY-SA 3.0, https://commons.wikimedia.org/w/index.php?curid=23867344.