Most transmission channels have a restricted bandwidth, either because of the limitations of the channel or because the bandwidth is shared between many users. Many forms of data have redundancy in it, thus if the redundant information was extracted the transmitted data would make better use of the bandwidth. This extraction of the information is normally achieved with compression.

When compressing data it is important to take into account three important characteristics: whether it is possible to lose elements of the data; the type of data that is being compressed; and how it is interpreted by the user. These three factors are normally interrelated. For example type of data normally defines whether it is possible to lose elements of the data. For computer data, normally if a single bit is lost, then all of the data is corrupted, and cannot be recovered. In an image, it is normally possible to lose data, but it could still contain the required information. For example, when we look at a photograph of someone, do we really care that every single pixel displays the correct color. In most cases the eye is much more sensitive to changes in brightness, and less sensitive to changes in color. Thus in compressing an image, we could actually reduce the amount of data in an image, by actually losing some of the original data. Compression which loses some of the elements of the data is defined as lossy compression, where lossless compression defines compression which does not lose any of the original data. Video and sound images are normally compressed with a lossy compression whereas computer-type data has a lossless compression. The basic definitions are:

Apart from lossy and lossless compression, an important parameter is the encoding of the data. This is normally classified into two main areas:

Entropy coding

Entropy coding does not take into account any of the characteristics of the data and treats all the bits in the same way. As it does not know which parts of the data can be lost, it produces lossless coding. As an example, imagine that you had a system which received results from sports matches, around the world. With this there would be no way of knowing the scores that would be expected, and the maximum size of the name that is stored for the result. For example a UK soccer match might have a result of Mulchester 3 Madchester 2, and the results of an American football match might be Smellmore Wanderers 60 Drinksome Wanderers 23. Thus, all the information would have to be stored as characters, where each character is stored in an ASCII format (such as with 8 bits for each character), with some way to delimit the end of the score. We could compress this, though, even not knowing the content of the result. For example there are repeated sequences in the scores: chester and Wanderers. These we could store each of these once, and then just make reference to it in some way. This technique of finding repeated sequences is a typical one in compression, and is used in ZIP compression (which is a general-purpose compression technique).

Typically entropy coding uses:

Source encoding.
Source encoding normally takes into account characteristics of the information. For example images normally contain many repetitive sequences, such as common pixel colors in neighboring pixels. This can be encoded as a special coding sequence. In video pictures, also, there are very few changes between one frame and the next. Thus typically the data encoded only stores the changes from one frame to the next. In our example of sports matches we could identify the type of sport, and then compress the data based on this. For example in a UK soccer match we could have a table of all the names of the sports clubs that we were storing the results for. Thus, for example, we may have 256 professional clubs, which would require 8 bits to store a reference value for each of these (number 0 to 256). So that 0 could be Mudchester, 1 could be Malchester, 2 could be Readyever United, and so on. We can also compress the scores, as we know that the goals scored for a team is very unlikely to be more than 31, thus we could use 5 bits to encode the score (0 to 31). Thus each score could be sent as an 8-bit reference for the home team, followed by a 5-bit value for their score, followed by an 8-bit reference for the away team, and finally by a 5-bit value for their score. Thus we only need 26 bits to store each of the scores, which is a large saving.

In fact we could compress the data even more, as we know that the most probable goals scored will be 0, 1 or 2. Thus we could store zero goals as 00 (in binary), one goal as 01 (binary), two goals as 10 (in binary), three goals as 110 (in binary), four goals as 1110 (in binary), and so on. Thus, as most scores will only have between zero and two goals scored for each team, the average number of bit will be just over 2 bits. For example if the scores were: 0, 1, 0, 2, 3, 2, 2, 1 and 1 (which would be stored as 00, 01, 00, 10, 110, 10, 10, 01 and 01), this would require 19 bits, which is an average of 2.11 bits. This is a reduction from 5 bits for each score. This technique is known as Huffman coding.