Re: [Phys-L] grades (was: no more SAT optional essay and subject tests)

[John Denker <jsd@av8n.com>]

On 1/19/21 9:54 PM, stefan jeglinski wrote:

> I often recount that in my career, when I was part of evaluating a
> new-hire, none of me or my colleagues EVER looked at grades – we did
> NOT care.

Yeah. That's how it is at places that know what they're doing, places
that trust their own processes, trust their own people to exercise
judgment.

Along the same lines, I often cite this example:

— At many universities, the music department offers a "music
appreciation for dummies" course. The students listen to music and
talk about it. Music majors are not allowed to take the course.
— This stands in contrast to the course in composition & orchestration
that is taken by music majors, by real musicians.

Suppose two physicists are applying for a job.  One got an A in music
appreciation, while the other got a C+ in music composition. I am a
*lot* more impressed by the latter.

That illustrates one of the fundamental moral hazards posed by the GPA
concept:
— In any given course, striving for a good grade is mostly correlated
with learning the material.
— In contrast, when /choosing/ courses, obsessing over GPA leads students
to choose the easiest course, not the one where they will learn the most.

Another moral hazard is when grade pressure tempts students to cheat.

> But these students live and die by them, and they are convinced that
> their futures depend on them. And at times I'm not sure I can fault
> them – it's simply not true that their grades won't be considered,
> perhaps strongly.

Yeah. Here's a story that illustrates why the students are right, and
the system is wrong. You can't entirely blame the students.

By law in New Jersey, Pennsylvania, and many other states, you are not
eligible for a teaching certificate unless you have a college GPA of
3.0 or better. Meanwhile, Princeton is known for rather competitive
admissions and lack of grade inflation. That means somebody who
graduates from Princeton with a B- in physics is permanently not
allowed to teach in Camden or Reading (where they desperately need
teachers).

> How do we start with this chicken and egg – do we attack the
> testing, or the emphasis on grades? Which begets which?

I'm not convinced either begets the other. They're both broken, but
broken in different ways for different reasons. Neither SAT nor GRE
scores are particularly well correlated with GPA.

The idea of a high-stakes summative test at the end of high school
(SAT) or the end of undergrad school (GRE) is "supposed" to solve (or
at least detect the problem mentioned above, i.e. choosing super-easy
courses, leading to maximal GPA with minimal learning.

> And I'm not one to say to toss it all out. I'm not smart enough to
> figure out a system that doesn't test and doesn't have grades – I'm
> not that inventive.

I'm not convinced that's the right goal, or even the right way to
frame the question. As I said yesterday, modifying a famous Obama
quote: "I am not opposed to all testing. I'm opposed to dumb testing."

The way forward is to replace bad tests with better tests, and a
better-balanced mixture of tests. People are fixated on mandatory
high-stakes summative tests that teachers hate, even while there is a
dire shortage of optional day-by-day formative testing resources that
teachers would love to have.

The GPA concept, strictly speaking, is not fixable. It is by
construction one dimensional, whereas learning and skill are
multi-dimensional. There is a famous theorem that says you cannot
change dimensionality in a way that is one-to-one and continuous. So a
single number cannot possibly be a reasonable way to measure
achievement or "merit" (whatever that means).

Anybody with any sense looks not at the GPA but the whole transcript.
I'm much more interested in which courses were taken than what the
grades were.

Even within a course, "total points" earned doesn't tell you what you
need to know. It's important check things sub-topic by sub-topic. By
the end of the program, each student should meet requirements in each
of the critical sub-topics. (This does *not* mean that all students
are required to learn everything in the same order before moving
on. Sal Khan goes waaaay overboard in this direction.) The spiral
approach is key here. Start by learning the basic outline; spiral back
and flesh things out later.

This is one of the good things about the FAA Airman Certification
Standards I mentioned yesterday. Everybody has to meet the standards
... but the flight instructor has complete discretion about how to get
students to that point.