This is not correct.
In nearly all statistical tests, 'significance' is a function of sample
size and the size of the underlying effect. Larger sample sizes make it
possible to reliably detect smaller effects (typically because larger
samples have smaller standard errors). The smaller the effect in the
population, the larger the sample size needed to detect the effect. By
'detect' here I mean find a statistically 'significant' difference.
With very large samples, very small actual differences can be reliably
detected (i.e., reported as 'significant', p < 0.05).
It's important to understand the difference between:
(a) the probability that two sample means (porportions, etc.) are from the
same population (i.e., the p-value), and
(b) the magnitude of the difference between sample means (i.e., the effect
size).
P-values are a function of both sample size and effect size.
Effect size is a function of the world (if you will) and is not affected by
sample size.
As scientists, we probably ought to be more concerned about effect sizes
and less concerned about statistical significance. When designing an
experiment, first decide what the smallest practically significant
difference would be in your comparisons. Then draw a sample large enough to
reliably detect differences of that size. If you find no difference in your
sample comparisons, it probably wasn't there in the population.
This is what power analysis is all about.
The best known discussion of these issues is Cohen's _Statistical Power
Analysis for the Social Sciences_, published by Erlbaum. Highly recommended.
Bruce L. Lambert, PhD
Department of Pharmacy Administration (M/C 871)
University of Illinois at Chicago
833 S. Wood St.
Chicago, IL 60612-7231