The rise of big data and the fall of statistical significance
Jerome Shimizu, Brett Gordon and Keith Kohrs InsightExpress and Columbia Business School
Big Data offers a lot of information, and so we can learn a lot from Big Data. However, Big Data sometimes requires us to change the way we learn. We now can capture information on a significant proportion of our population of interest. In these instances we should no longer use the standard statistics based on an infinite population size. Instead, we should use the Finite Population Correction (FPC) factor.
Suppose we are interested...