Lecture 10 - 2025 / 4 / 22

Encoding

$\{0, 1\}^4 \to C = \text{Null}(H)$

$\quad S \quad \mapsto SG$

standard form $G = (I_4, P_{4\times 3})$

Decoding

$x \mapsto Hx \begin{cases} =0 & x \mapsto x\\ \ne 0 & \text{which column of } H \end{cases}$

Extending

$[7, 4, 3] \xrightarrow{\rm entend} [8, 4, 4]$

$[m, k, d \text{ (odd)}] \xrightarrow{\rm entend} [m + 1, k, d + 1 \text{ (even)}]$

Hamming $[7, 4, 3]$ code is equivalent to the code of the following generator matrix (Cyclic code)

$G = \begin{bmatrix} 1101000\\0110100\\0011010\\0001101 \end{bmatrix}$, "shift 1 bit, shift 1 bit, shift 1 bit"

Channel Capacity

Raw message $\xrightarrow{\rm Source\ coding}$ Source code $\xrightarrow{\rm Channel\ coding}$ Channel Code $\xrightarrow{\rm Noisy \ channel}$ ...(Decoding)...

1. How to formulate a noisy channel: $X \xrightarrow{P(Y | X)} Y$

2. Given a noisy channel $P(Y | X)$, how much information the receiver obtain about $X$?

$I(X; Y) = D(P_{XY} \| P_X P_Y)$

Definition (Channel Capacity): The channel capacity for a noisy channel $P(Y | X)$ is $C := \max_{P_X} I(X; Y)$.

It is possible that $H(Y) > H(X)$.

Channel Coding Theorem (Informal)

$R :$ rate, number of bits transmitted each time. e.g. $0 \to 00000, 1\to 11111$

if $R < C$, $\exists$ error correcting code, $\textit{error} \to 0$
if $R > C$, $\nexists$ error correcting code, $\textit{error} \to 0$