Skip to content

Chapter 6 of 7

Quantitative Research and Measurement

Quantitative research measures scale, frequency, and statistical confidence. Surveys are the most common instrument and rely on a set of design principles to reduce bias: questions should be short, specific, behaviorally anchored, single-barreled, balanced in wording, and pre-tested with the target audience, with option order randomized for non-scaled items. A "double-barreled" question, such as "Was the support agent friendly and knowledgeable?", asks two things at once and should be split. Common response biases shape survey design. Acquiescence bias is the tendency to agree with statements regardless of content and is countered by reversing some items or using forced-choice formats. Social desirability bias leads respondents to over-report approved behaviors and under-report stigmatized ones; anonymous, indirect, or behaviorally anchored questions reduce it. Non-response bias arises when those who answer differ systematically from those who do not, so heavy non-response can render results misleading even if the questions are perfect. The Likert scale, commonly five- or seven-point, asks respondents to indicate agreement, frequency, or satisfaction; even-numbered scales force a direction, while odd-numbered scales allow neutrality.

Three widely used customer metrics deserve special mention. Net Promoter Score (NPS) asks how likely a customer is to recommend the product on a 0–10 scale and is calculated as the percentage of Promoters (9–10) minus the percentage of Detractors (0–6), used as a proxy for loyalty and growth. Customer Satisfaction Score (CSAT) is typically the percentage of respondents who rate a specific interaction as satisfied or better, captured right after the event. Customer Effort Score (CES) asks how easy it was for the customer to get something done; lower effort is associated with higher retention and referral. Churn is the percentage of customers (or revenue) lost in a period, while retention is the percentage kept, two sides of the same coin, and the most useful research explains which behaviors precede churn. A churn survey, sent to customers who have canceled or stopped using the product, asks the reason and what could have changed their decision; it is useful when the response is voluntary and unbiased and when paired with behavioral data. A "smoking gun" signal in churn analysis is a specific behavioral or attitudinal pattern that sharply raises the probability of churn within a defined window, such as login frequency dropping below once a week in a SaaS product.

Several quantitative methods deserve attention for product and pricing decisions. Conjoint analysis asks respondents to choose between bundles of attributes at varying levels, statistically inferring the relative importance and trade-off value of each attribute. MaxDiff, or best-worst scaling, asks respondents to pick the most and least important items from a small set across many sets to produce robust, ratio-scaled importance scores. Van Westendorp price sensitivity analysis asks four questions (too cheap, cheap but not unrealistic, expensive but not unrealistic, too expensive) to map a range of acceptable prices for a product. The Willcox test for new product concepts asks respondents to rate a written concept on purchase intent, uniqueness, value for money, and believability, then segments the audience by enthusiasm to identify the most receptive early customers. Survey design should also keep separate "stated importance," what respondents say matters, from "derived importance," calculated from their actual trade-off behavior; the two frequently disagree, and derived importance is usually a better predictor.

Statistical literacy underpins quantitative conclusions. A confidence interval is a range derived from sample data that is likely to contain the true population value at a stated confidence level, commonly 90% or 95%. Margin of error is the plus-or-minus figure that describes how much survey results are expected to vary from the true population value at that level. A p-value is the probability, assuming the null hypothesis is true, of observing data as extreme as what was collected; by convention, p < 0.05 is often used as a threshold for declaring statistical significance. Statistical significance, however, is not the same as practical significance: the former means an observed difference is unlikely due to chance, while the latter asks whether the effect is large enough to matter for the business, the user, or the design decision. An A/B test is a controlled experiment where users are randomly assigned to two or more variants of a product, message, or flow, and a defined outcome is compared. Different research questions call for different designs: exploratory research investigates an unclear problem to generate hypotheses, descriptive research measures who, what, when, and where, and causal research tests whether one variable actually causes a change in another, usually via experiment.

All chapters
  1. 1Foundations of Customer Research
  2. 2Qualitative Research Methods
  3. 3Sampling, Recruitment, and Ethics
  4. 4Customer Understanding Frameworks
  5. 5Synthesis and Insight Generation
  6. 6Quantitative Research and Measurement
  7. 7Sharing Research and Driving Decisions

Drill it

Reading is not remembering. These come from the Customer Research deck:

Q

What is customer research?

Customer research is the disciplined process of learning how customers think, behave, and make decisions so teams can design better products and messaging.

Q

Why is customer research valuable?

It reduces guesswork by grounding decisions in real customer evidence instead of assumptions.

Q

What is the difference between qualitative and quantitative research?

Qualitative research explores motivations and patterns in depth, while quantitative research measures scale, frequency, and statistical confidence.

Q

What is a customer interview?

A structured conversation used to understand needs, context, workflows, frustrations, and decision criteria.