The rise of big data and the fall of statistical significance

This paper argues that as a result of big data researchers should reconsider how they measure confidence intervals, and move from a calculation based on an infinite population size to one of 'Finite Population Correction' (FPC).

The rise of big data and the fall of statistical significance

Jerome Shimizu, Brett Gordon and Keith Kohrs InsightExpress and Columbia Business School

Big Data offers a lot of information, and so we can learn a lot from Big Data. However, Big Data sometimes requires us to change the way we learn. We now can capture information on a significant proportion of our population of interest. In these instances we should no longer use the standard statistics based on an infinite population size. Instead, we should use the Finite Population Correction (FPC) factor.

Suppose we are interested...

Not a subscriber?

Schedule your live demo with our team today

WARC helps you to plan, create and deliver more effective marketing

  • Prove your case and back-up your idea

  • Get expert guidance on strategic challenges

  • Tackle current and emerging marketing themes

We’re long-term subscribers to WARC and it’s a tool we use extensively. We use it to source case studies and best practice for the purposes of internal training, as well as for putting persuasive cases to clients. In compiling a recent case for long-term, sustained investment in brand, we were able to support key marketing principles with numerous case studies sourced from WARC. It helped bring what could have been a relatively dry deck to life with recognisable brand successes from across a broad number of categories. It’s incredibly efficient to have such a wealth of insight in one place.

Insights Team
Bray Leino

You’re in good company

We work with 80% of Forbes' most valuable brands* and 80% of the world's top top-of-the-class agencies.

* Top 10 brands