Thermodynamics of Computation: Landauer's Principle and LLM Inference

2026-07-30

Thermodynamics of Computation: Landauer’s Principle and LLM Inference

v1.2 | 2026-07-30 | Status: Peer-review ready

Landauer’s Principle

Statement: The erasure of one bit of information in a computational process must generate at least kT ln(2) heat, where k is Boltzmann’s constant and T is the temperature of the system.

Significance: This establishes a fundamental lower bound on the energy cost of computation. Information and entropy are deeply connected.

Calculation: At room temperature (300K):

kT ln(2) ≈ 2.85 × 10⁻²¹ J per bit
For 1 trillion parameters: ~2.85 × 10⁻¹² J per inference step
This is negligible compared to actual LLM energy costs

Application to LLM Inference

The Information Processing View

A single LLM inference pass:

Input tokens: ~n bits of information
Model state: ~m bits (parameters, activations)
Output tokens: ~p bits of information

From Landauer’s perspective, the erasure cost matters, not the computation cost. But LLM inference is largely information-preserving (no erasure in the thermodynamic sense).

Where the Energy Actually Goes

The actual energy cost of LLM inference comes from:

Matrix multiplications: O(n²) or O(n³) operations for transformer blocks
Memory bandwidth: Loading parameters and activations from DRAM
GPU inefficiencies: Not all ALUs are utilized at all times
Cooling overhead: Data center PUE (Power Usage Effectiveness) of 1.1-2.0

For a 70B parameter model:

Parameters: ~140 GB (BF16)
Activations: ~40-80 GB per sequence
Memory bandwidth: ~1 TB/s on A100
Power: ~300-700W per GPU

Energy per token: ~0.1-1 J per token (empirical measurements)

The Gap Between Theory and Practice

The gap between Landauer’s limit and actual energy use is enormous:

Landauer limit (per bit):   ~3 × 10⁻²¹ J
Actual (per token):        ~0.5 J
Ratio:                     ~10¹⁸

This gap comes from:

Reversibility: Most computation is irreversible (dissipates energy)
Hardware overhead: Transistors are far from ideal
Algorithm inefficiency: We don’t use reversible computing
Physical constraints: Signal propagation, leakage current

Implications for AI Efficiency

Theoretical Limits

If we could approach Landauer’s limit:

1 trillion tokens could be processed for ~0.4 J
This is about the energy of a single peanut
Current state: ~500,000 J (500 kJ) for 1T tokens
Efficiency gap: ~1 million times

Practical Paths Forward

Reversible computing: Theoretical but not yet practical at scale
Better algorithms: More efficient attention mechanisms
Hardware specialization: ASICs for LLM inference (like Google TPU)
Architectural innovations: Mixture of experts, sparse attention
Precision reduction: INT8, INT4, binary networks

Connection to AI Systems Work

Understanding these thermodynamic limits helps in:

Energy-aware model design: Choosing architectures based on compute efficiency
Hardware selection: Matching models to efficient hardware
Deployment strategy: Edge vs. cloud based on energy costs
Future-proofing: Understanding what’s physically possible

The principles that govern thermodynamic efficiency also apply to information efficiency. Good AI systems, like good physical systems, minimize waste.