Shannon (unit)
Units of information |
Information-theoretic |
---|
Data storage |
Quantum information |
The shannon (symbol: Sh) is a
The shannon also serves as a unit of the
Nevertheless, the term bits of information or simply bits is more often heard, even in the fields of information and communication theory, rather than shannons; just saying bits can therefore be ambiguous. Using the unit shannon is an explicit reference to a quantity of information content, information entropy or channel capacity, and is not restricted to binary data,[2] whereas "bits" can as well refer to the number of binary symbols involved, as is the term used in fields such as data processing.
Similar units
The shannon is connected through constants of proportionality to two other units of information:[3]
The hartley, a seldom-used unit, is named after Ralph Hartley, an electronics engineer interested in the capacity of communications channels. Although of a more limited nature, his early work, preceding that of Shannon, makes him recognized also as a pioneer of information theory. Just as the shannon describes the maximum possible information capacity of a binary symbol, the hartley describes the information that can be contained in a 10-ary symbol, that is, a digit value in the range 0 to 9 when the a priori probability of each value is 1/10. The conversion factor quoted above is given by log10(2).
In mathematical expressions, the nat is a more natural unit of information, but 1 nat does not correspond to a case in which all possibilities are equiprobable, unlike with the shannon and hartley. In each case, formulae for the quantification of information capacity or entropy involve taking the logarithm of an expression involving probabilities. If base-2 logarithms are employed, the result is expressed in shannons, if base-10 (common logarithms) then the result is in hartleys, and if natural logarithms (base e), the result is in nats. For instance, the information capacity of a 16-bit sequence (achieved when all 65536 possible sequences are equally probable) is given by log(65536), thus log10(65536) Hart ≈ 4.82 Hart, loge(65536) nat ≈ 11.09 nat, or log2(65536) Sh = 16 Sh.
Information measures
In information theory and derivative fields such as coding theory, one cannot quantify the 'information' in a single message (sequence of symbols) out of context, but rather a reference is made to the model of a channel (such as bit error rate) or to the underlying statistics of an information source. There are thus various measures of or related to information, all of which may use the shannon as a unit.[citation needed]
For instance, in the above example, a 16-bit channel could be said to have a
References
- ^ Since the information associated with an event outcome that has a priori probability p, e.g. that a single given data bit takes the value 0, is given by H = −log p, and p can lie anywhere in the range 0 < p ≤ 1, the information content can lie anywhere in the range 0 ≤ H < ∞.
- ^ Olivier Rioul (2018). "This is IT: A primer on Shannon's entropy and Information" (PDF). L'Information, Séminaire Poincaré. XXIII: 43–77. Retrieved 2021-05-23.
The Système International d'unités recommends the use of the shannon (Sh) as the information unit in place of the bit to distinguish the amount of information from the quantity of data that may be used to represent this information. Thus, according to the SI standard, H(X) should actually be expressed in shannons. The entropy of one bit lies between 0 and 1 Sh.
- ^ "IEC 80000-13:2008". International Organization for Standardization. Retrieved 21 July 2013.