Question 1

Define Information.

Accepted Answer

Information is mathematically defined as the absolute measure of uncertainty resolved by the occurrence of a specific, probabilistic event.

Question 2

What is Entropy?

Accepted Answer

Entropy is the rigorous mathematical calculation of the absolute average information content or uncertainty present in a discrete random variable.

Question 3

Define Channel Capacity.

Accepted Answer

Channel Capacity is the mathematically proven, absolute maximum rate at which information can be reliably transmitted over a noisy communication channel.

Question 4

What is Mutual Information?

Accepted Answer

Mutual Information mathematically quantifies the exact amount of information that one random variable physically reveals about another variable.

Question 5

Define Source Coding.

Accepted Answer

Source Coding is the mathematical algorithmic process of violently compressing data to absolutely minimize the average number of bits transmitted.

Question 6

What is Huffman Coding?

Accepted Answer

Huffman Coding is a mathematically optimal, prefix-free source coding algorithm that aggressively assigns the shortest binary codes to the most frequent symbols.

Question 7

Define Shannon's Theorem.

Accepted Answer

Shannon's Theorem mathematically proves that absolute error-free communication is physically possible over a noisy channel if the transmission rate is below capacity.

Question 8

What is Error Detection?

Accepted Answer

Error Detection is the architectural protocol of mathematically appending redundant bits to a message to identify catastrophic corruption during transit.

Question 9

Define Parity Check.

Accepted Answer

A Parity Check is the most primitive mathematical error detection architecture, utilizing a single redundant bit to enforce an even or odd sum.

Question 10

What is a Linear Block Code?

Accepted Answer

A Linear Block Code is an advanced algebraic coding architecture where any mathematical linear combination of valid codewords generates another valid codeword.

Question 11

Explain the concept of Self-Information and Entropy with examples.

Accepted Answer

A rigorous mathematical deconstruction of Self-Information and Entropy, explaining how logarithmic functions strictly quantify uncertainty, surprise, and the theoretical limits of data compression.

Question 12

Describe the Shannon-Fano coding technique.

Accepted Answer

A detailed exploration of the Shannon-Fano Coding technique, demonstrating its top-down mathematical architecture for generating prefix-free variable-length codes based on symbol probability.

Question 13

Explain the concept of Channel Capacity and Shannon's Channel Coding Theorem.

Accepted Answer

An advanced theoretical analysis of Channel Capacity, explicitly focusing on the Shannon-Hartley theorem which defines the absolute mathematical boundaries of data transmission over an AWGN channel.

Question 14

Discuss the Hamming Code for error detection and correction.

Accepted Answer

A detailed mathematical breakdown of the Hamming Code, explaining its strict structural use of interleaved parity bits to both detect and completely correct single-bit hardware flips during transmission.

Question 15

Explain the concept of Cyclic Redundancy Check (CRC).

Accepted Answer

A technical exploration of the Cyclic Redundancy Check (CRC), detailing its highly robust polynomial division architecture designed to detect catastrophic burst errors in network data frames.

Question 16

Describe the properties of Linear Block Codes (Generator Matrix, Parity Check…

Accepted Answer

A rigorous algebraic analysis of Linear Block Codes, detailing how the Generator Matrix (G) constructs codewords and how the Parity Check Matrix (H) aggressively executes syndrome decoding.

Question 17

Explain the concept of Convolutional Codes.

Accepted Answer

An architectural breakdown of Convolutional Codes, explaining how their memory-based shift registers violently intertwine continuous data streams for superior error correction via Viterbi decoding.

Question 18

(a) Explain the concept of Entropy and its properties in detail. (b) Calculat…

Accepted Answer

An exhaustive mathematical exposition on Entropy, rigorously defining its formulas, properties, and calculating strict probability vectors, followed by the architectural relationships of Joint and Conditional Entropy.

Question 19

(a) Explain Huffman Coding algorithm in detail. (b) Construct Huffman codes f…

Accepted Answer

A definitive architectural breakdown of Huffman Coding, mathematically demonstrating its bottom-up tree construction to achieve absolute optimal prefix-free compression, including a full efficiency calculation.

Question 20

(a) Explain the different types of communication channels (BSC, BEC). (b) Dis…

Accepted Answer

A rigorous technical breakdown of Discrete Memoryless Channels (BSC and BEC), coupled with an aggressive mathematical derivation of Shannon's Theorem and the calculation of channel capacity in noisy environments.

Question 21

(a) Explain Linear Block Codes in detail. (b) Discuss the Hamming bound and p…

Accepted Answer

A catastrophic mathematical deconstruction of Linear Block Codes and Hamming Codes. Explains the rigid Generator matrices, mathematically proves the Hamming bound for perfect codes, and executes a full (7,4) error detection and correction cycle.

Question 22

(a) Explain Cyclic Codes in detail. (b) Construct a cyclic code using generat…

Accepted Answer

A deep architectural dive into Cyclic Codes and BCH Codes. Demonstrates the strict modulo-2 polynomial division utilized by the Generator Polynomial to mathematically encode and decode data streams without complex matrix multiplication.

Information Theory and Coding