The Feldwick Factor: Is creative and planner intuition a better predictor of effectiveness than pre-testing?
Do we know exactly how predictive any pre-testing method is? There's little information in the public domain that lets us make an objective estimate of their success rates, or what levels of false negatives or false positives they generate.
But I find reasons to believe that even the best of them fall short of the high predictive standards that they implicitly lead us to expect. The dominant methods are based on models of message transmission and conscious attention that have been seriously challenged, and we also know that there is a negative correlation between IPA Effectiveness Awards success and the use of pre-testing, especially quantitative.
So I would agree that pre-testing, as practised, is much more fallible than its proponents would like us to believe. But I'm not quite convinced by your alternative formula, either. Let me try to explain why.