Re: Corpora: Statistical significance of tagging differences

Steve Finch (steve.finch@thomson.com)
Sat, 20 Mar 1999 10:54:25 -0500

In message <199903200440.WAA06172@email1.cc.uic.edu>, "Bruce L. Lambert" writes

>As scientists, we probably ought to be more concerned about effect sizes
>and less concerned about statistical significance. When designing an
>experiment, first decide what the smallest practically significant
>difference would be in your comparisons. Then draw a sample large enough to
>reliably detect differences of that size. If you find no difference in your
>sample comparisons, it probably wasn't there in the population.

Trouble is that in POS tagging, if the goal is to get a sentence (or
phrase) right, even a small difference in error rates on tags can
multiply up to a large difference in accuracy on entire sentences or
phrases.

- S