Entropy and Analog to Digital Conversion

If I use an A/D converter to convert an analog voltage into 8 digital bits, have I changed the entropy of the system? Has the entropy of the analog voltage changed? Surely info has been lost in the conversion, but somehow the digital version seems more "ordered." What about when I apply those 8 bits to a D/A? The analog result is going to be an approximation of the original, but has the entropy changed? It's a kind of reversibility, but not exact. Does the concept of entropy even apply to such a system?
- Alan Becker (age 67)
Coopersburg, PA, USA

Those are great questions because they address just the issues people usually run into with the idea of entropy.

First, it's easy to answer all questions about changing entropy: the total entropy went up in the process. It sounds like the "system" here is meant to be a pretty complete description of the interacting parts, so yes the system entropy also went up?

What about the impression of how much a system is "ordered"? That turns out to be a very fuzzy word. Sometimes our sense of order corresponds to reduced entropy and sometimes it doesn't. The physical law (2nd law of thermodynamics) about entropy always increasing is about entropy, not our sense of order. For example, when a strongly supercooled thermally isolated cup of water starts to freeze, it forms a lot of regular ice crystals. It seems more ordered, but the entropy of course went up. That's because it became a bit warmer, and on a microscopic scale things are jiggling around more.

The same idea applies to your A/D and D/A.  The particular features you're keeping track of (the voltage, the average state of a few switches) have a tiny bit of entropy associated with them.  Little changes in the temperature of the transistors etc.are far larger. That's why it can look like entropy is decreasing if you only keep track of the tiny parts.

A similar issue is raised in many creationist tracts which claim that life decreases entropy. They mean that a few features corresponding to obvious large-scale organization seem more "ordered" in living things. If you count all the little chemical changes etc., however, life increases entropy just like everything else does.

Mike W.

(published on 09/11/2015)