Lecture 9 - 2025 / 4 / 15

Message $\to$ Error Corrrecting Code
$\{0,1\}^m \to \quad \{0, 1\}^n (n\ge m)$

1. $\forall c_i, c_j \in \{0, 1\}^n$, $d_H(c_i,c_j) \ge 2t + 1$ bits.
2. Sphere packing bound $2^n / \sum_{i=0}^{t} \binom n i\ge 2^m$
3. Gilbert-Vashamov bound
4. Efficient Encoding / Decoding Algorithm

Hamming Code

$H = \begin{bmatrix}0001111\\0110011\\1010101 \end{bmatrix}$, note that the columns are $1234567$ respectively.

For $x_i, x_j \in \text{Null}(H) = \{x \in \{0,1\}^7, Hx = 0\}$

$d_H(x_i, x_j) = \|x_i - x_j\|_1 = \|x_k\|_1$, $x_k \in \text{Null}(H)$

$H \cdot x_k = 0 \Rightarrow \| x_k \|_1 \ge 3$ (weight)

For Hamming $(7, 4, 3)$ Code (Linear code), sphere packing bound is tight, which are called perfect code.

Decoding

Correct 1-bit error $c_i \in \text{Null}(H) \to x \to c_i$, $d_H(x, c_i) \le 1$ bit

$Hx =\begin{cases} 0 & x \to c_i\\ H(c_i + \delta) = H\delta & x - \delta \to c_i\end{cases}$

Encoding

$m_i \in \{0,1\}^4 \to c_i = [m_i, \cdot, \cdot, \cdot]$

$g_1, g_2, g_3, g_4 \in \text{Null}(H)$, let $G = (g_1, g_2, g_3, g_4)$ then $H\cdot G = 0$.

$c \in \text{Null}(H) = \sum_{i=1}^{4} a_i g_i = G \cdot a$, where $a = (a_1, a_2, a_3, a_4)^\top \in \{0, 1\}^4$

$\{0, 1\}^4 \to \text{Null}(H)$
$\quad a \quad \mapsto \quad G\cdot a$

We must have a standard form of $G$ s.t. $Ga = \begin{bmatrix} I_4 \\ G_3 \end{bmatrix}a = \begin{bmatrix} a \\ G_3 a \end{bmatrix}$

Why? Permute $H$ to $[3567, I_3]$ then $H \cdot G = 0$ must has a solution $G = \begin{bmatrix} I_4 \\ G_3 \end{bmatrix}$.

General steps for Linear code $[n, k, d]$ ($k$-dim subspace of $\{0,1\}^n$):

1. Generating matrix $G_{n\times k}$.
2. the column vector of $G$ is a basis of the $k$-dim subspaces.
3. Encoding $m \mapsto Gm$.
4. $\exists$ standard form of $G = \begin{bmatrix} I_k \\G_{n-k}\end{bmatrix}$.

Nonlinear Code

$(n, k, d)$

Shortening. Given $(n, m, d)$ code. How to change $[7, 4, 3]$ to $(8, 4, 4)$?