This is because every data set must contain some chance patterns which are not be present in the population under study, or simply disappear with a sufficiently large sample size. This activity was formerly known in the statistical community as data mining, but that term is now in widespread use with an essentially positive meaning, so the pejorative term data dredging is now used instead.Ĭonventional statistical procedure is to formulate a research hypothesis, (such as 'people in higher social classes live longer') then collect relevant data, then carry out a statistical significance test to see whether the results could be due to the effects of chance.Ī key point is that every hypothesis must be tested with evidence that was not used in constructing the hypothesis.

( September 2007) ( Learn how and when to remove this template message)ĭata dredging ( data fishing, data snooping) is the inappropriate (sometimes deliberately so) search for 'statistically significant' relationships in large quantities of data. Unsourced material may be challenged and removed. Please help improve this article by adding citations to reliable sources.

This article needs additional citations for verification.

Data dredging example