GreenBook provided a 'Sneak Peek' of the findings from their latest GRIT (Research Industry Trends) survey in a webinar on May 14th, with a panel of research sector leaders to discuss the key points.

According to the survey, the biggest challenge is around technology. At the heart of the findings, and of the discussion, was the decades old dilemma of what the market research sector should be. Our heritage, and much of our skill-set, expertise and experience is vested in methodologies for collecting primary data to provide fresh insights into consumer, and citizen, behaviour. Not simply to identify the 'who', 'what', 'where' and 'when', but importantly the 'why'.

However, our clients seem to be busy with analytics and data integration in the 'new' world of big data, questioning what we might bring to the party. This is, of course, NOT a new finding – it emerged in the 1980s as database marketing enabled clients to undertake their own 'research' by either recruiting analysts, or turning to the new breed of marketing analytical companies that were getting off the ground.

So what are we expected to do with technology? We've harnessed it to cut costs and time out of the survey process, to the delight of the client. Some on the panel thought more automation might help, giving more time (and a bigger share of the budget) to interpretation and telling a persuasive story to the client. But history tells us that this may be wishful thinking, as all it will do is create demands for further economies and yet cheaper research products.

Technically, the whole online survey process can be automated, right through from sample selection, questionnaire design, coding, analysis and feeding the findings straight into the client's intranet or dashboard. Ditto for 'phone surveys (but not necessarily legally possible). What more can we do? The choices seem to be quality/value versus cheaper/faster/good enough versus customised/full service/traditional.

In my view we could be poking ourselves in the eye with a fork if we believe we can beat the client on cost if its in-house analytics versus survey research, a point made by Bob Shaw back in the 1980s when he predicted the demise of market research as clients sought to maximise the return on their investment in the customer database. Of course it hasn't turned out quite like that, but the research sector fought shy of entering the then new world of analytics (DunnHumby now up for sale by Tesco for £bn), and despite some advances, researchers still have their noses pressed against the outside of the steamy shop window, envious of what's going on inside, but not invited by the shopkeeper to enter.

If clients are investing heavily in data, and the tools/skills to analyse it, will they want help from us, and would we be the natural first call, anyway? One panel member thought we should stop pushing clients to do the data synthesis themselves and do this ourselves – but at what cost (to the client), and with what skills to offer? The latest findings show that the sector's focus is still on primary research, whilst the client's focus is on data and insight. But, as one panel member reminded us, even the technological and information giant, Google, launched a survey division…

We still have an identity crisis – we don't seem to know who we are, and more importantly, what we want to become.

And currently, we're a bit on the back-foot in the UK on our ability to do the traditional day job successfully, the pollsters having been criticised for the accuracy of the published opinion polls for the UK's 2015 general election, plus being blamed in some quarters, mostly the media, for contributing to the eventual outcome by influencing the development of policies, the party strategies and public opinion (maybe outside of war time Britain doesn't like coalition governments, and people carried that view into the polling booth – just a thought). What is the impact of this on our case to clients: 'trust me, I'm a market researcher' in trying to persuade them to let us into their shop to sell them on the idea we have the skills and tools they need?

But there are other sides to technology, creating new opportunities to collect data in real-time for example, and we are only at the start of the mobile revolution.

Many of the new players cited as being innovative in the latest survey offer technology based solutions and products. But I'll finish with the panel's discussion as to why BrainJuicer tops the innovation ratings by such a high margin, and for the fourth year running. What does this tell us about what the sector needs to do? Their understanding and applying new knowledge from the field of neuroscience, System 1 and System 2 thinking, was seen as a primary factor.

But other strengths highlighted by the panel were inspirational leadership, recruiting good people, being provocative, engaging with clients especially when presenting findings, being visionary, prepared to talk about what they do, and, good at marketing. Straight out of the business textbooks – success based on leadership, understanding customer needs, innovatory products and having the capabilities to leverage them. Note, however, no mention of technological prowess!

The full report on the latest GRIT survey will be published in June (www.greenbook.org).