A couple of months ago, I wrote about the need for marketers and their partners to close the increasingly widening gap between commercial need and creative solution by stopping specialising ourselves into irrelevance. This month I want to focus on the other gap that I see driving a wedge between what we do and its effectiveness: the gap between us and real, visceral customer understanding and empathy.

Ten years ago, Bain published a report entitled Closing the Delivery Gap. In it was a damning piece of evidence: 80% of CEOs believed their companies delivered a superior customer experience, yet only 8% of their customers felt the same way. A staggering tenfold difference. And things aren't getting any better. Two years ago, Forrester published research that showed that when it came to delivering better customer experience innovations, 58% looked at what competitors in their category were doing, 72% looked at what companies outside their category were doing, 62% utilised technological advancement, and 53% looked for customer empathy. So, to close a gap with their customers, the last place companies are looking is to better understand their customer. Nearly one in two companies didn't even bother looking there. This is, to steal Ted Levitt's phrase from 50 years ago, marketing myopia.

Now you may argue that when $11 billion is spent annually on market research in the US alone, then surely we are investing in understanding the customer. But we all know the limitations and superficial nature of most of this type of research - research that is designed to validate and quantify things we already know or, at the very least, common sense would tell us. It's hard to have true empathy with your customers if they are behind a two-way mirror or, worse still, your understanding is based on whatever 'insight' you can glean from a quick Google search. Worse still, with everyone asking the same questions in the same way, research became one of the biggest drags on innovation as everyone rushed to the middle of the road.

More and more companies are sensitive to this and so we do research that is more ethnographic and observational in nature. Under A.G. Lafley, Procter & Gamble famously made its senior management and brand managers spend time every year living with their customers. More promisingly still, the spread of UX best practice and its maxim that people involved in designing products should spend two hours every six weeks with the people they are designing for, has begun to increase the frequency and depth of our customer interaction and increased the likelihood of us designing marketing that has good customer understanding.

However, perhaps the biggest thing we could do to design better experiences for people is to remove the gap between us and them. Rather than simply observing people, what if we decided to spend some time living like them? Facebook this month launched an internal programme called '2G Tuesdays'. Employees who opt in will be switched to a simulated 2G connection for an hour, mimicking the experience that millions of people have with Facebook every day. With profiles, pages, pictures and videos taking longer to load, employees will feel the reality of the product experience for many outside the developed world. The hope is that by closing the empathy gap for those who don't have direct experience of emerging markets, it will trigger new ideas on how to improve the product experience for this group.

Time will tell what product initiatives come from this but it seems a novel and better way to close the experience gap between those making and marketing experiences and those experiencing them. Perhaps we could take this to the extreme and begin to encourage those involved in the design of experiences – whether in product, marketing or one of their partners – to live like their customers, not just observe them. I'd certainly be intrigued to see what all the immensely creative people in Silicon Valley would do if they lived like their typical user in a developed, let alone developing, market. By truly removing the gap, perhaps we would see less lipstick being applied to pigs and a little less solving of 'first world problems' and, instead, make it more likely that we create experiences that genuinely make lives a little better and move society on.

By truly removing the gap, perhaps we would see less lipstick being applied to pigs and a little less solving of 'first world problems' and, instead, make it more likely that we create experiences that genuinely make lives a little better and move society on.