It could be just a coincidence. These things happen. If you're
operating at the 5% significance level then, on average, 5% of the
t-tests that you do on uncorrelated data will erroneously state that
there is a relationship.
This becomes especially problematic when you dabble with data mining.
For example, supposing I have 7 dependent variables that are all
random and uncorrelated. Nevertheless, I start looking for
correlations between all possible pairs of these 7 variables at the 5%
significance level. That's 21 pairs of variables, so I can expect to
find an average of 5% of 21 = 1.05 'significant' correlations.
Other than that, there's the whole issue of what you mean by
'practical'. If medical trials show that a new drug prolongs life at
the 0.001% confidence interval, that's a very strong statistical
result. But if the amount it prolongs your life by is only 1%, and
makes you feel miserable while you're taking it, who cares? |