Re: Cooked Data

["John S. Denker" <jsd@MONMOUTH.COM>]

At 11:46 AM 8/26/99 -0500, brian whatcott wrote:
> the exclusion of
> outriders is not a sneaky-pete issue, but an element of a most
> respectable way of handling experimental data statistically.
>
> It can be shown that the principal result of using median like values
> instead of mean like values is to require rather more numbers in the
> sample.

That can be shown only under special circumstances.

> Justifications are available for excluding rather large proportions
> of data points at the skirts in rigorous analyses.
> Needless to say, there is a right way.

There is also a wrong way.  When students in the lab exclude outliers, they
are almost certainly doing it the wrong way.

Consider the following example:  Suppose you are the assayer for a
gold-mining company.  The prospectors bring you a huge sample-set.  Now it
turns out that the average density of gold near the earth's surface is of
very little interest to the company.  The only points that matter are the
outliers.  I suggest you don't throw them out.