Re: [Phys-L] Entropy

[John Denker <jsd@av8n.com>]

On 8/9/24 11:16 AM, Marc Reif via Phys-l wrote:

> I'm not going to get very far talking about probability spaces to
> kids who don't know anything about probability.

That's why physics teachers get the big (??) bucks. The students are
supposed to arrive knowing algebra and trigonometry, but they don't,
so it is necessary to "review" all that, by which I mean teach it
almost ab_initio. It's the same story with probability, except it's
easier since they don't need to know very much probability. Mostly
they just need to count states.

(According to the modern (post-1933) notion of probability, counting
states qualifies as a probability measure, but you don't need to
emphasize that.)

> So it appears there is no way to explain entropy to students in
> their first algebra-based physics course that is both basically
> correct and understandable to students at that level.

It's not that bad. I would approach entropy via the back door, i.e.
via *information* . Rationale: For one thing, these days there are
more jobs in information technology that there are in the steam-engine
industry. Secondly, information has direct application to design of
experiments, which is applicable to everything from astrophysics to
zoology. The "12 coins puzzle" makes this point quite nicely. Thirdly,
you can mention in passing that entropy is the amount of information
you /don't/ have.

For the next level of detail on this, see Part 2 below.

> Disorder doesn't quite get it, because some apparently disordered
> systems spontaneously order themselves (the salad dressing example)
> while increasing entropy.

Agreed. Disorder is a bad idea at the most fundamental level. For one
thing, disorder, insofar as it can be quantified at all, is a property
of the microstate, whereas entropy is a property of the macrostate.
Entropy is an ensemble average, namely the expectation value of the
surprisal.

> Energy dissipation doesn't get it, because it's possible to take
> energy in macroscopic systems and "dissipate it" into a smaller
> volume within the system as thermal energy.

Agreed. Information and entropy are defined without reference to
energy. Some systems have a well-defined temperature and some don't,
but they all have a well-defined entropy and they all uphold the
second law of thermodynamics.

Without a temperature you have no hope of connecting energy to
entropy via dE = T dS + .....

> What about invoking "irreversibility"? Is that a tool that works to
> describe increasing entropy?

Well, that is a partial description of entropy, but only partial, and
there's a lot more to the story. Lots of things move information and
entropy around /reversibly/ or nearly so.

///////////////////

Part 2:

As I see it, a reasonable goal for an introductory unit is getting
the students to understand the 12 coins puzzle.
  https://av8n.com/physics/twelve-coins.htm

That's partly a lighthearted game or puzzle, but it's also genuine
industrial-strength information theory and design of experiment. To
win the game you have to design the experiment to use information very
efficiently. You need to distinguish 24 states, and three trits of
information can distinguish at most 27 states, even with perfect
efficiency.

Another classic is the 20 questions game. This has the advantage that
many people have already heard of it. OTOH they may not be familiar
with the mad scientist approach to playing the game, which wins every
time.
  https://av8n.com/physics/twenty-questions.htm

It may also come as a surprise that there is a /parallel/ algorithm
for winning. That is to say, you can write down the questions in
advance and answer them in any order. This highlights the absolute
purity of the information involved.

In the spirit of pedagogical stepping stones, you can play some
ultra-simple card games, with gradually increasing complexity. Start
with a mini-deck consisting of two cards, the ace and the two. Shuffle
them and deal them onto the table. How many yes/no questions must you
ask to identify the ace? One. How many to identify both cards? Also
one, total, since you get the last one for free by process of
elimination.

Now use three cards. How many yes/no questions to identify the ace?
Two. How many to identify all the cards? Three.

Now four cards. Remarkably enough, it still takes only two questions
to identify the ace, if you're clever about it. So five questions to
identify all the cards.

Now five cards. How many yes/no questions to identify the ace? Three.
How many to identify all the cards? You might guess eight, but in fact
it can be done in seven, if you're clever about it. Hint: 5 factorial
is 120, and 2⁷ is 128.

The entropy of a well-shuffled 52-card deck (considering only rank and
suit) is just shy of 226 bits. The thermodynamic entropy of a cup of
water is multiple *moles* of bits.

Here's an amusing story about symmetry and information, absolutely
pure information, independent of any physical mechanism:
  https://www.av8n.com/physics/clock.htm