Lecture 8 - 2025 / 4 / 8

Channel Coding

Motivation:

  1. Noisy channel
  2. Algorithm reduces (corrects) error

Reptition Code

Decoding: Nearest Neighbor

Channel Coding

Lower bound: {0,1}m→{0,1}n\{0, 1\}^m \to \{0, 1\}^n, Goal: dH(ci,cj)≥2t+1d_H(c_i, c_j) \ge 2t + 1 (bits) correct tt bits error

Given t,mt, m, estimate a lower bound of nn

Sphere packing bound: m≥2n/∑i=0t(ni)m \ge 2^n / \sum_{i=0}^{t} \binom{n}{i}

Upper bound: {0,1}m→{0,1}n\{0, 1\}^m \to \{0, 1\}^n, Goal: dH(ci,cj)≥δnd_H(c_i, c_j) \ge \delta n (bits)

Given m,δ∈(0,1/2)m, \delta \in (0, 1/2), find nn such that there must exists c1,⋯ ,c2m∈{0,1}nc_1, \cdots, c_{2^m} \in \{0, 1\}^n s.t. dH(ci,cj)≥δn,∀i≠jd_H(c_i, c_j) \ge \delta n, \forall i \ne j

Probabilistic method: c1,c2∼U{0,1}nc_1, c_2 \sim U \{0, 1\}^n, P(dH(c1,c2)≥δn)=1−∑i=0δn(ni)/2nP(d_H(c_1, c_2) \ge \delta n) = 1 - \sum_{i=0}^{\delta n} \binom{n}{i} / 2^n

Proof sketch:

  1. Uniformly generatic c1,⋯ ,c2mc_1, \cdots, c_{2^m}, forall i≠ji \ne j, P(dH(ci,cj)<δn)<e−Θ(n)P(d_H(c_i, c_j) < \delta n) < e^{-\Theta(n)}

  2. P(∃i≠j,dH(ci,cj)<δn)<(2m2)e−Θ(n)P(\exists i \ne j, d_{H}(c_i, c_j) < \delta n) < \binom{2^m}{2} e^{-\Theta(n)}

  3. (2m2)e−Θ(n)<1\binom{2^m}{2} e^{-\Theta(n)} < 1

Gilbert-Vashamov bound: n≥cmn \ge c m for some constant cc, ∃c1,⋯ ,c2m\exists c_1, \cdots, c_{2^m}, dH(ci,cj)≥δnd_H(c_i, c_j) \ge \delta n.

Hamming Code

H∈Mat(GF(2)),Null(H)={x∣Hx=0}H \in {\rm Mat(GF(2))}, \textit{Null}(H) = \{x \mid Hx = 0\}

What if ∀x,y∈Null(H)\forall x , y \in \textit{Null}(H) ?