Shannon’s Source Coding Theorem

ShannonThe idea of Shannon’s famous source coding theorem is to encode only typical messages. Since the typical messages form a tiny subset of all possible messages, we need less resources to encode them. We will show that the probability for the occurence of non-typical strings tends to zero in the limit of large message lengths. Thus we have the paradoxical situation that although we „forget“ to encode most messages, we loose no information in the limit of very long strings. In fact, we make use of redundancy, i.e. we do not encode „unnecessary“ information represented by strings which almost never occur.

Kommentare sind geschlossen