Channel Capacity — Nyquist and Shannon

Two limits on how fast a channel can carry bits, why you always compute both, and which one is actually binding.

The idea

The question underneath this whole topic is one Claude Shannon answered in 1948: given a channel that distorts and adds noise, can you still send data with an error rate approaching zero?

His answer was yes — if and only if your rate stays below a threshold. Below that threshold, error probability can be pushed towards zero. Above it, no amount of cleverness helps. That threshold is the channel capacity.

There are two separate ways a channel can run out of room, which is why you end up with two formulas rather than one.

Bandwidth limits how fast the signal can change. Noise limits how finely you can tell one signal level from another. Nyquist handles the first, Shannon the second, and the real channel is stuck with whichever bites first.

The vocabulary first

Two pairs of terms get confused constantly, and questions are built on the confusion.

Data element vs signal element. A data element is the smallest unit of information — a bit. A signal element is the shortest unit of the signal, the thing that carries data elements. The lecture’s analogy: the signal element is the vehicle, the data element is the number of people in it, and vehicles come in different sizes.

Data rate vs signal rate. Data rate is bits per second (bps). Signal rate — also called baud rate, pulse rate or modulation rate — is signal elements per second, measured in baud. Signal rate is what defines bandwidth.

That gives the engineering goal in one line: push the data rate up while keeping the signal rate down. More bits per vehicle. That ratio is spectral efficiency, and every modulation scheme in the next topic is an attempt at it.

Nyquist — the noiseless limit

How it works

Bit rate = 2B log2(L)

Bandwidth B limits how many signal elements per second you can push through: 2B of them. That part is fixed by physics.

What is not fixed is how much each one carries. Use L distinguishable levels and each element carries log2(L) bits. Multiply:

$\text{Bit rate} = 2B \log_2(L)$

Two levels gives one bit per element. Four gives two. Sixteen gives four.

The lecture’s note on the noiseless case is the important one: there is no limit on the number of levels you may encode, but you are still limited by B. So on a genuinely noiseless channel you could use a million levels and go as fast as you like.

Real channels are not noiseless, which is what Shannon is for.

Shannon — the noisy limit

How it works

C = B log2(1 + P/N)

$C = B \log_2\!\left(1 + \frac{P}{N}\right)$

P is received signal power, N is noise power. Notice what is not in this formula: the number of signal levels. Shannon does not care how you encode. Whatever you do, you cannot beat this.

The intuition for why noise caps you: levels have to be far enough apart that noise cannot push one into another. More noise, fewer usable levels, less information per element.

And the case that catches people — if N = 0, then C is infinite. The lecture states this outright. It is not a broken formula; it is Shannon saying he has nothing to tell you about a noiseless channel. Nyquist takes over there.

Where marks get lost

Decibels are not a ratio

P/N in Shannon’s formula is a power ratio. Questions often hand you decibels instead, and you must convert:

$\frac{S}{N} = 10^{(\mathrm{dB}/10)}$

Ten, not twenty. The 20 log10 form is for amplitude and voltage ratios. Using it here turns 30 dB into 31.6 instead of 1000, and every number after that is wrong.

A quick sanity check: 10 dB is a ratio of 10, 20 dB is 100, 30 dB is 1000. Each 10 dB multiplies by ten.

Using both

Worked example

1 MHz bandwidth, SNR 63

This is the lecture’s own example, and it shows why you compute both.

Shannon first — what is the ceiling?
$C = B \log_2(1 + P/N) = 10^6 \times \log_2(1 + 63)$
$= 10^6 \times \log_2(64) = 10^6 \times 6 = 6 \text{ Mbps}$
No encoding scheme can beat 6 Mbps on this channel.
Nyquist second — what must I build to reach it?
$\text{Bit rate} = 2B \log_2(L)$
Set the bit rate to the 6 Mbps Shannon allows:
$6 \times 10^6 = 2 \times 10^6 \times \log_2(L)$
$\log_2(L) = 3 \quad\Rightarrow\quad L = 8$
Eight signal levels, each carrying 3 bits.

The two formulas answer different questions. Shannon says how fast is possible. Nyquist says what you have to build to get there.

AnswerC = 6 Mbps, requiring L = 8 levels

Aside

Notice the SNR was 63, not 64. Exam questions pick numbers so that 1 + SNR lands on a power of two and the logarithm comes out whole. An SNR one below a power of two is a strong hint you are meant to do it in your head.

Try it

Two things worth doing with it. Fix the bandwidth and raise the SNR — watch Shannon climb while Nyquist sits still, because Nyquist does not know noise exists. Then fix the SNR and raise the level count — watch Nyquist climb past Shannon and stop mattering, because you have hit the noise wall.

Wherever the two cross is where a real system is designed to sit.

Check yourself

A channel has 3 kHz of bandwidth and an SNR of 30 dB. You are using 4 signal levels. What rate do you get, and what is stopping you going faster?

Convert first: 30 dB is a ratio of 1000. Shannon gives about 3000 x log2(1001) which is roughly 29.9 kbps. Nyquist with 4 levels gives 2 x 3000 x 2 = 12 kbps. Nyquist binds — the noise would allow far more, but you have only chosen 4 levels. Adding levels helps here. Once you pass about 16 levels, Shannon takes over and adding more stops helping.

In the exam

Both formulas, verbatim, with every variable named. They appear on the formula sheet, and a question that asks you to state one is free marks.
Always compute both. A question giving you bandwidth, SNR and a level count wants both numbers and a statement of which binds.
dB conversion. 10^(dB/10). The most common arithmetic slip in the whole module.
The noiseless case. Shannon goes infinite; Nyquist still limits you via B. Asked as a “what if” almost every year.
Working backwards. Given a target rate and a bandwidth, find L. Rearrange Nyquist — and remember L must be a whole number, and in practice a power of two.
Data rate vs signal rate. Define both with units, and say that signal rate determines bandwidth.