Weighing up qualitative and quantitative research

Stephen Connell, MORPACE International, shows how qualitative tracking can help build customer partnerships

This article looks at the ways in which both types of research evidence are used in the business-to-business side of the telecommunications industry. And it makes a case for giving qualitative information an equal role in an increasingly numerate and quantitative culture; and for reconsidering respondent anonymity as a default option in market research.

The respondents from our research are engineers, marketers and business planners in telephone companies and operators (Cellnet, Orange and Telewest are some of the UK companies interviewed) and the client is Nokia Networks, which supplies, installs and maintains the infrastructure that is the basis of the fixed and mobile services offered. morpace International is currently embarking on the fourth year of a programme of customer satisfaction studies with about 100 of Nokia Networks major corporate customers, each spending millions of euros each year with the company. Interviewing is carried out by a programme of depth interviews, in person, in over 30 countries in Europe, the Asia/Pacific region and the Americas.

This research and the business it examines are an extreme example of the way in which relationship marketing is affecting Code of Conduct issues for market researchers. Respondents are offered anonymity (at the start and at the end of the interview) but fewer than one in 20 take it, either because I am not saying anything I havent told them or because they positively want their remarks to be attributed. These interviews are another part of the relationship, a communication link from the customer straight to the top management of a global company who read all the reports and to a host of engineers and designers who are represented by the account managers with whom respondents deal directly. The implications of this approach for other parts of the telecommunications industry and for other sectors will be considered at the end of this article.

Customer satisfaction

The world of customer satisfaction research is well documented and there is no shortage of publications, or points of view, about methods of interviewing, analytical techniques or approaches to interpretation. Some of the groundwork has been done by the survey research industry but the self-styled Quality Movement has contributed a great deal. In both cases there is a strong and increasing emphasis on moving from issues of satisfaction to those of loyalty: it is a truism that satisfied customers are often disloyal they can be seduced away from their favourite airline or computer vendor by pricing discounts or other promotions and that single incidents of perceivedly inadequate service can result in a change of supplier by a hitherto loyal customer.

In these circumstances, there is a hunt for survey measures that will predict or anticipate brand or vendor switching, that will identify individual customers or client cohorts that are at risk. Since prediction is inherently statistical, there has been a strong focus on quantitative analysis and on the identification of key metrics. Numerical scores are also widely used in compensation plans for sales/account management and, in some cases, for board directors.

For reasons that will become clear, I believe that there are markets in which metrics of this kind provide one of the key inputs to the essential processes of ensuring that companies are properly attuned to the needs of their customers. Telecommunications is one good example.

The telecommunications industry

Nokia Networks and its competitors (Lucent, Ericsson, Motorola, Marconi, Alcatel, Cisco, Siemens and many others) provide the platforms on which voice and data services operate, supplying optical fibres and transmission equipment, radio base stations, public and private exchanges (or switches) and a wide variety of software tools and databases that are used to manage networks and provide services.

Their customers the fixed and mobile telecommunications operators are dealing with much greater uncertainty than hitherto: many of them are making enormous investments. For example, the DTI is forecast to raise anything from 500 million to 5 billion from the current auction for five licences for Third-Generation mobile telephony services, at a time when the potential demand and revenues for such concepts as mobile videotelephony and the transmission of still images by cellular telephone are unknown.

The decision on how rapidly to abandon traditional telephone-industry network infrastructures based largely on the telephony concept of a connection that is established for the duration of a call or session and to move to the Internet protocol (IP), which is based on the intermittent transmission of packets or cells of data is a similarly radical choice.

We are transforming our business to support online activity. We really have a situation, in the transformation, in which data becomes the core of our business with telephony remaining a very important revenue stream for a long time to come, it wont necessarily be the core driver as it was in the past. Source: A national telephone company in the Asia-Pacific region.

In some cases these choices are being made; elsewhere they are being resisted. The point is that they cannot be ignored and that there are no clear guidelines to business success.

Some of the key uncertainties for the telecommunications operators are:

  • decisions about future size, scale and geographic coverage (Vodafones geographic expansion is a case in point)
  • how far they should be developing partnerships with content providers (for example, with organisations like CNN, Reuters or the BBC, as part of a move into mobile information provision) rather than concentrating on pure network and service provision
  • how to manage the introduction, in a highly competitive market, of new and more powerful data services that will cannibalise existing service revenues (for example, from ISDN)
  • which technologies to use in mobile data, in the backbone networks that carry Internet traffic, and in the provision of broadband services to the home
  • choice of a vendor who has the financial and technical strength to deliver appropriate products to the future market at the right time (nobody wants a vendor whose products will be late to market or inappropriately featured and/or priced).

All of this means that telecommunications carriers are becoming increasingly demanding and are looking for business partners rather than vendors. Companies like Nokia are being expected to:

  • contribute to the development of business cases for new services and the investment that they will require
  • provide a clear vision of the ways in which their technical strategy and product development programme will provide the basis for the customer to move into new ways of working and creating revenue
  • bring appropriately priced and featured products to the market at the right time.

Customer satisfaction depends, therefore, on more than providing good or excellent services and products: it is increasingly a matter, in this industry, of inspiring confidence that the right choices are being made for the future.

Assessing customer requirements

Our research for Nokia Networks utilises a scoring approach in the context of the depth interview: 35 aspects of service provision, grouped under five processes (for example, sell, deliver) are used as measures of the respondents satisfaction with Nokias performance. On average, three to six interviews per account are carried out and average scores and ranges are presented at a variety of levels individual accounts, regions, divisions and a global set of overall findings. This quantification provides important year-on-year comparisons and acts as a set of signposts, pointing the way to areas of excellence and underperformance. It is, however, the qualitative information obtained through detailed probing and discussion that provides the basis for action:

l it is the basis for recommendations at each of the reporting levels

l extensive use of quotations, from named organisations and individuals, provides vivid illustrations of key issues and compelling accounts of areas in which future business could be at risk

l in database form, it allows quality experts in a variety of manufacturing and service disciplines to identify the loci and causes of dissatisfaction.

Much of this information is analysed and reported by us, the survey researchers, but managers within Nokia Networks, who understand the history of the account and the technical context, will take the analysis a stage further. All of this information has to be attributed to the customer organisation (if this permission is not given, the interview is cancelled or terminated), and it is more useful when it is attributed to the individual respondent (as it normally is).

Customer satisfaction the impossible dream

There is, however, another very important reason why qualitative information of a very detailed kind is important in the very demanding world of telecommunications service provision: it is almost impossible fully to satisfy these customers now, and it will get increasingly harder to achieve high scores. The uncertainties, risks and the pace of change in the telecommunications industry mean that fixed and mobile operators are becoming and will become much more demanding customers. A high score today is no guarantee of satisfaction qualitatively: it is merely a recognition that Nokia, or any company rated, is on a par with leading competitors.

Almost all of the customers who give generally high satisfaction scores for each issue have a number of concerns. These quotations about key improvement areas for Nokia Networks are from respondents who have all given a score of 8 or better for satisfaction with products and services. (The scoring uses a 010 scale where 0 = very dissatisfied, 5 = ok, 10 = very satisfied.)

In europe

It is important for Nokia to help us make a success of our marketing, to find new markets an area in which they can make useful inputs.

Help me move into a new era of operations. A new era in terms of vastly increased scale, vastly increased complexity of services; and at the same time pull cost down. More involvement in our strategy.

In the americas

Transform themselves into a partner when you become partners you rely on one another, you support each other, you anticipate each others needs and expectations, and you try and fulfil them before you are even requested to do so not because of the interest that may derive from it but because of the well-being that may derive from it on behalf of the whole partnership.

It is very important to keep us informed, not just in the traditional sense of marketing to grow the business we depend on having more options incorporated in the network differentiating ourselves from competition ... to be successful we need a system that offers the options before the clients demand them.

In asia/pacific

Help us to target higher-usage subscribers be more innovative in ways of helping us increase market share.

Learn more about our network so they can be more proactive in suggesting new equipment and solutions.

Ultimately, to commit each year in advance to steep cost reductions.

These insights into customer requirements are, as described above, studied and perused intensively. They tend to change from year to year, as the regulatory and competitive climate for customer organisations evolves, and as new technologies appear and mature. In contrast the quantitative trends are not dissimilar to the invented graph shown in Exhibit 1.

The overall trend is slightly upward and the results are generally satisfactory, but it is the underlying issues that are most important. Low scores clearly signpost underperformance but areas, regions or divisions with excellent standards of service tend to get fairly flat graphs, partly because some individuals will never give scores of 9 or 10 (however good the standards of service), partly because things do go wrong (occasionally); but mainly for the reasons already given: these customers expect to get excellent service and they still want more.

An extended relationship

This mixture of qualitative and quantitative information comes from a customer satisfaction measurement programme that is embedded in the business planning of Nokia Networks, and that expands the communication channel from the customer to the parts of the organisation that users and buyers of Nokia equipment do not normally see: the engineers and designers who develop future products, top management who set strategies and priorities, quality and process managers who strive for improvement and consistency of service, and many more.

All of this puts the market research agency in a special role: we deal and pass on attributed information, we work with and submit information to Nokia offices around the globe and we provide highly disaggregated data, some of which is raw or unanalysed and some of which is analysis of very specific issues, or of the circumstances in a single customer organisation.

As the introduction to this article makes clear, this category of customer has the spend to justify such treatment, but, more and more, customer satisfaction research is likely to deal with the individual customer in consumer and business-to-business markets rather than with the sample, the segment or the cohort.

Just as direct marketing is becoming increasingly personalised, so customer satisfaction research will become more and more concerned with disaggregated information and with the provision of data about individual customers. In these circumstances a number of issues are raised:

  • customer satisfaction is about more than satisfaction it covers faith in the vendor and their ability to continue to offer appropriate products
  • market researchers will not automatically conduct research on the basis of respondent anonymity
  • large-scale satisfaction measurement programmes will have an increasing qualitative component.

These issues, illustrated in the context of one industry, should be understood by the designers of survey programmes and of industry Codes of Conduct, if the needs of rapidly changing industries are to be met.



Kathrin Haarstick

Stephen Connell

    Stephen Connell is the managing director of MORPACE International Ltd, which he co-founded in 1997. He has been a market researcher and forecaster in the telecoms industry since the early 1970s, with such companies as Rank Xerox, Gordon Simmonds Research, IFC Research and FDS International.

A Visual Way to Explore Brand Imagery