Question
Download Solution PDFWhat is the entropy of a communication system that consists of six messages with probabilities 1/8, 1/8, 1/8, 1/8, 1/4, and 1/4 respectively ?
Answer (Detailed Solution Below)
Detailed Solution
Download Solution PDFConcept:
The entropy of a probability distribution is the average or the amount of information when drawing from a probability distribution.
It is calculated as:
\(H=\underset{i=1}{\overset{n}{\mathop \sum }}\,{{p}_{i}}{{\log }_{2}}\left( \frac{1}{{{p}_{i}}} \right)bits/symbol\)
pi is the probability of the occurrence of a symbol.
Calculation:
The entropy will be:
\(H=4\times\frac{1}{8}log_2(8)+2\times\frac{1}{4}log_2(4)\)
\(H=\frac{4\times3}{8}+\frac{2\times2}{4}\)
H = 2.5 bits/message
Last updated on Jul 2, 2025
-> ESE Mains 2025 exam date has been released. As per the schedule, UPSC IES Mains exam 2025 will be conducted on August 10.
-> UPSC ESE result 2025 has been released. Candidates can download the ESE prelims result PDF from here.
-> UPSC ESE admit card 2025 for the prelims exam has been released.
-> The UPSC IES Prelims 2025 will be held on 8th June 2025.
-> The selection process includes a Prelims and a Mains Examination, followed by a Personality Test/Interview.
-> Candidates should attempt the UPSC IES mock tests to increase their efficiency. The UPSC IES previous year papers can be downloaded here.