Entropy
Entropy
Definition: A measure of the disorder, randomness, or number of possible microscopic arrangements (microstates) in a system. In an isolated system, total entropy tends to increase over time.
How It Works
- The second law of thermodynamics states that the total entropy of an isolated system never decreases; it stays constant for an idealized reversible process and increases for any real, irreversible process.
- Entropy increase reflects a system evolving toward the macrostate with the largest number of possible microscopic arrangements, not because the system “wants” disorder, but because that outcome is overwhelmingly more statistically likely.
- There are two equivalent definitions: the statistical definition, entropy as a function of the number of microstates, and the classical thermodynamic definition, entropy as heat transferred reversibly divided by temperature.
- Heat flows spontaneously from hot to cold, and never the reverse, because that direction increases the total entropy of the two objects combined.
- Entropy is a state function: it depends only on a system’s current state, not on the path taken to reach it, even though calculating a real (irreversible) process’s entropy change requires imagining an equivalent reversible path.
- Local entropy can decrease, water freezing into ordered ice crystals, a refrigerator cooling its contents, a living cell organizing molecules, but only by increasing entropy elsewhere by an equal or greater amount.
- Entropy has a close cousin in information theory: Shannon entropy measures uncertainty or information content in a message using the same logarithmic form as Boltzmann’s formula.
- Because entropy consistently increases toward the future, it gives the universe a thermodynamic “arrow of time,” distinguishing past from future macroscopically even though the underlying microscopic laws of physics are symmetric in time.
- The third law of thermodynamics adds a boundary condition: the entropy of a perfect crystal approaches zero as temperature approaches absolute zero, since only one microstate remains available.
Entropy Change in Common Processes
| Process | ΔS system | ΔS surroundings | ΔS universe |
|---|---|---|---|
| Free expansion of gas into vacuum | Positive | ~0 | Positive |
| Reversible isothermal gas expansion | Positive | Negative | ~0 (ideal limit) |
| Heat flow from hot to cold object | Negative (hot loses) | Positive (cold gains more, weighted by 1/T) | Positive |
| Water freezing into ice | Negative | Positive | Positive |
| Living organism maintaining order | Negative (locally) | Positive (via metabolic heat) | Positive |
| Mixing two different gases | Positive (mixing increases microstates) | ~0 | Positive |
Illustration
Under the Hood
Key equations:
S = k_B ln(W) (Boltzmann's entropy formula, k_B = 1.381 × 10⁻²³ J/K)ΔS = ∫ dQ_rev / T (thermodynamic/Clausius definition)ΔS = Q / T (isothermal reversible process)ΔS_universe = ΔS_system + ΔS_surroundings ≥ 0Worked Example 1: Entropy change of melting ice Given: m = 0.5 kg of ice melting at 0°C (273 K), latent heat of fusion L_f = 334,000 J/kg Step 1: Heat absorbed: Q = mL_f = 0.5 × 334,000 = 167,000 J Step 2: ΔS = Q/T = 167,000/273 Answer: ΔS ≈ 611.7 J/K, the entropy gained by the water as it melts at constant temperature.
Worked Example 2: Isothermal expansion of an ideal gas Given: n = 1 mole of gas, isothermal reversible expansion doubling its volume, R = 8.314 J/(mol·K) Step 1: ΔS = nR ln(V₂/V₁) = 1 × 8.314 × ln(2) Answer: ΔS = 8.314 × 0.693 ≈ 5.76 J/K, the entropy gained purely from the gas occupying more possible positions.
Worked Example 3: Counting microstates (Boltzmann’s formula) Given: a simplified system of 10 fair coins, where each arrangement of heads/tails is a distinct microstate Step 1: Total microstates: W = 2¹⁰ = 1024 Step 2: S = k_B ln(W) = (1.381 × 10⁻²³) × ln(1024) Answer: S ≈ (1.381 × 10⁻²³)(6.93) ≈ 9.57 × 10⁻²³ J/K, illustrating how entropy scales with the logarithm of the number of equivalent configurations, even though a real thermodynamic system involves far more particles.
Worked Example 4: Heat flow between two reservoirs Given: Q = 1000 J flows from a hot reservoir at T_h = 400 K to a cold reservoir at T_c = 300 K Step 1: Entropy lost by the hot reservoir: ΔS_hot = −Q/T_h = −1000/400 = −2.5 J/K Step 2: Entropy gained by the cold reservoir: ΔS_cold = +Q/T_c = +1000/300 ≈ +3.33 J/K Step 3: Net change: ΔS_universe = −2.5 + 3.33 Answer: ΔS_universe ≈ +0.83 J/K > 0, confirming heat flows spontaneously in this direction, consistent with the second law.
Worked Example 5: Carnot efficiency limit Given: an ideal heat engine operating between T_h = 500 K and T_c = 300 K Step 1: Maximum possible efficiency: e = 1 − T_c/T_h = 1 − 300/500 Answer: e = 1 − 0.6 = 0.4 = 40%. No real engine operating between these two temperatures can exceed 40% efficiency, a direct consequence of entropy never decreasing in a reversible cycle.
Worked Example 6: Entropy cost of erasing information (Landauer’s principle) Given: erasing 1 bit of information at room temperature T = 300 K, k_B = 1.381 × 10⁻²³ J/K Step 1: Minimum entropy generated per bit erased: ΔS_min = k_B ln(2) Step 2: ΔS_min = (1.381 × 10⁻²³)(0.693) ≈ 9.57 × 10⁻²⁴ J/K Step 3: Minimum heat that must be dissipated: Q_min = T × ΔS_min = 300 × 9.57 × 10⁻²⁴ Answer: Q_min ≈ 2.87 × 10⁻²¹ J per bit erased, an almost immeasurably small but strictly nonzero energy cost that sets a fundamental physical limit on how efficient computing can ever become.
Why It Matters
- Entropy sets a hard theoretical ceiling on the efficiency of every heat engine, from car engines to power plant turbines, no matter how well engineered.
- Refrigeration and air conditioning require external work specifically because moving heat from cold to hot decreases entropy locally, which must be paid for by a larger entropy increase elsewhere.
- Landauer’s principle in computing states that erasing one bit of information has a minimum unavoidable entropy cost, linking information theory directly to physical thermodynamics.
- Cosmologists use entropy to describe the long-term fate of the universe, trending toward a maximum-entropy “heat death” where no usable energy gradients remain.
- Entropy explains why so many everyday processes are irreversible: you can scramble an egg but never unscramble it, because the scrambled state has vastly more possible microscopic arrangements.
- Chemical engineers use entropy alongside enthalpy (via Gibbs free energy) to predict whether a reaction will proceed spontaneously at a given temperature.
- Materials scientists track entropy changes during phase transitions, melting, boiling, and crystallization, to design alloys and predict processing behavior.
- Weather and climate models incorporate entropy production to understand how energy disperses through the atmosphere and oceans over time.
Common Pitfalls
- Treating entropy as literally the same thing as a messy room; that’s a loose analogy. Entropy is a precise statistical quantity tied to the number of equivalent microscopic configurations.
- Believing entropy can never decrease anywhere; only the total entropy of an isolated system (or the universe) never decreases. Local decreases happen constantly.
- Confusing entropy with energy; entropy isn’t a form of energy, it’s a measure related to how energy and matter are distributed among possible states.
- Assuming living organisms or evolution violate the second law; organisms are open systems that export entropy to their surroundings (as metabolic heat) faster than they reduce their own internal entropy.
- Forgetting that ΔS = Q/T requires a reversible heat transfer path; using this formula directly on an irreversible process’s actual path gives the wrong answer.
- Making sign errors between heat leaving a system (negative Q) and heat entering (positive Q) when computing entropy change.
- Conflating Boltzmann’s statistical entropy with Shannon’s information entropy without noting they measure conceptually different things, even though the math looks nearly identical.
- Assuming a perfectly efficient (100%) heat engine or refrigerator is just an engineering challenge; it’s ruled out by entropy considerations regardless of how advanced the technology becomes.
Comparison
| Type of entropy | Formula | Units | Field |
|---|---|---|---|
| Thermodynamic (Clausius) | dS = dQ_rev/T | J/K | Classical thermodynamics |
| Statistical (Boltzmann) | S = k_B ln(W) | J/K | Statistical mechanics |
| Information (Shannon) | H = −Σ p_i log₂(p_i) | bits | Information theory |
| Black hole (Bekenstein-Hawking) | S = k_B A c³/(4Gℏ) | J/K | General relativity / quantum gravity |
FAQ
Can entropy ever decrease? Locally, yes, constantly. Globally, for a truly isolated system or the universe as a whole, no; total entropy never decreases.
Does life violate the second law of thermodynamics? No. Living organisms maintain low internal entropy only by continuously exporting entropy to their environment, mainly as waste heat, so the total entropy of organism plus environment still rises.
Does entropy apply to black holes? Yes. Black holes have a well-defined entropy proportional to their event horizon’s surface area, discovered by Bekenstein and Hawking, and merging two black holes always increases the total horizon area, consistent with the second law.
What is the “heat death” of the universe? A hypothetical far-future state of maximum entropy, where energy is spread so evenly that no temperature differences or usable energy gradients remain to do any work.
Is entropy the same thing as chaos? Not precisely. It’s better described as a count of equivalent ways a system’s microscopic details can be arranged while still looking the same macroscopically, “disorder” is just an informal shorthand for that idea.
Why can’t we simply reverse time to undo entropy increase? The underlying laws of motion for individual particles are time-symmetric, but the sheer improbability of a high-entropy state spontaneously reorganizing into a low-entropy one makes reversal practically impossible for any system with more than a handful of particles.
Example
A running car engine converts the chemical energy in gasoline into mechanical work, but no engine converts all of that energy into useful motion. Some is always rejected as waste heat through the exhaust and radiator, a direct and unavoidable consequence of entropy: a heat engine cannot achieve 100% efficiency without violating the second law of thermodynamics.
Related Terms
Referenced by