Key takeaways
- Generative research explores the problem before a solution exists; evaluative research tests a concept, prototype or live product with users.
- Read the verb in the research question: what or why points to generative work, can or how well points to evaluative work.
- Evaluative research holds two kinds of study, formative evaluation during design and summative evaluation at or after launch, per Nielsen Norman Group.
- The research question, not the method, sets the mode: interviews, surveys and card sorts can serve either side.
- In our program, generative fieldwork (7–15 days) comes before evaluative studies (4–8 days), inside 4–8 weeks.
Generative vs evaluative research comes down to one question you can answer from the wording of a ticket: is there a solution to put in front of users yet? Two illustrative tickets for an expense-reporting app show the test. "Find out why field managers file expense reports a week late" opens with why and has no screen to show anyone, so it calls for generative research. "Check whether managers can submit a receipt in the new capture screen unaided" names a screen and asks can, so it calls for evaluative research.
Both tickets belong to user experience (UX) research, yet they need different participants, different questions and a different deliverable. Inside our UX research services program, generative fieldwork is scheduled ahead of evaluative studies, because a test can only check a problem someone has already framed.
Generative vs evaluative research, what is the difference?
Generative research explores the problem to find what to build, while evaluative research tests a solution, whether a concept, prototype or live product, to see if it works for users.
Both are goals of user research, also called UX research. Generative work studies the problem space, meaning people's pain points and mental models. The user research types chapter of the User Interviews UX research field guide also names it discovery research, exploratory research or foundational research. Evaluative work studies the solution space; UserTesting calls it evaluation research.
Some teams prefer the discovery vs validation research framing: discover needs, then validate a design with users. Erika Hall, co-founder of Mule Design Studio, reduces the split to two questions that UserTesting quotes: "What problem might we solve?" and "How well is our solution working?"
The table compares generative vs evaluative research on eight criteria, with our program's step lengths in the last row.
| Criterion | Generative research | Evaluative research |
|---|---|---|
| Research question | What do users need, and why? | Does this design work, and how well? |
| Product stage | Before a solution exists | Once a concept, prototype or product exists |
| Artifact needed | None | A concept, prototype or live product |
| Data type | Mostly qualitative | Qualitative and quantitative |
| Question framing | Open-ended exploration | A testable hypothesis |
| Typical methods | User interviews, contextual inquiry, diary studies | Usability testing, tree testing, closed card sorting |
| Main output | Personas, journey maps, opportunity areas | Findings, task success, fixes |
| Decision output | A direction to pursue | Keep, change or drop |
| Our program | Generative fieldwork (7–15 days) | Evaluative studies (4–8 days) |
The product stage and the artifact needed settle most debates: with no concept, prototype or product to show, there is nothing to evaluate yet.
What does generative research mean?
Generative research is user research that builds an understanding of people's needs, behaviors and pain points before any solution exists, so a team knows which problem is worth solving.
Its main output is a set of opportunity areas, not a verdict on a screen. For the expense ticket, that means sitting with managers on a Friday afternoon to see why receipts pile up.
What does evaluative research mean?
Evaluative research is user research that measures how well a concept, prototype or live product meets user needs, and it repeats through iterative design as the solution changes.
Each round checks a hypothesis, such as "managers find the capture button without help", and its decision output is keep, change or drop. On the expense ticket, it starts the day a clickable prototype exists.
Is generative research the same as generative AI?
No, generative research is a user research mode that studies people and their problems, while generative artificial intelligence (AI) is a class of models that produce text, images or code.
Generative design is algorithmic shape-making in computer-aided design (CAD) software, and program evaluation in public health is a separate field. The formative and summative labels come from education and instructional design, where Michael Scriven coined them in 1967, as Alita Kendrick's Nielsen Norman Group (NN/g) article records. AI tools speed up transcription and tagging in a generative study; our UX research methods guide answers whether AI replaces UX research. To sort your own research questions, run the verb test in the next section.
Are you exploring the problem or testing a solution?
The generative vs evaluative research decision starts with one check, whether a solution exists to put in front of users: if not, explore the problem; if so, test it.
Then read the verb. Questions that open with what, why or "how do users" are generative, because they explore the problem space. Questions that open with can, does, "how well" or "which version" are evaluative, because they test the solution space.
Explore the problem (generative research)
- Why do field managers file expense reports late?
- What do managers do with paper receipts on a trip?
- How do finance admins chase missing reports today?
- What makes a manager trust an expense total?
Test a solution (evaluative research)
- Can managers submit a receipt in the new capture screen unaided?
- Does the Expenses menu put "Add receipt" where managers look?
- How well does the approval screen work on a phone?
- Which of two report summaries do admins read faster?
Generative research fits a new product with no user interviews behind it, a new market, a feature people ignore for reasons analytics cannot show, and teams with no shared personas or journey maps. Evaluative research fits a redesign with untested navigation and information architecture, a prototype before build, and a launch decision.
When a design exists but nobody knows whether it solves the right problem, run a short generative round first. Write your next three research questions down now and circle each verb.
Which methods are generative research methods?
Generative research methods are open-ended studies of people and their work, such as user interviews, contextual inquiry, diary studies and participatory design, run before anyone fixes a solution.
- User interviews: 10–15 interview participants per user group, so patterns repeat before anyone trusts them.
- Contextual inquiry: watching users work in their real setting, 4–6 users in our program, because people skip steps they no longer notice.
- Field studies and ethnographic studies: longer observation, for behavior that only shows over time or in a group.
- Diary studies: participants log their days for weeks, so routines surface.
- Participatory design: users sketch their ideal tool, which reveals priorities.
- Open card sorting: users create their own groupings of content before a menu exists.
- Jobs to be Done interviews: why someone chose a product for a job, so needs can be ranked.
In our program, generative fieldwork (7–15 days) is where researchers hold recorded user interviews and contextual inquiry sessions. Many teams cluster notes in affinity diagrams; ours are tagged in Dovetail and become personas, journey maps and a Jobs to be Done opportunity map. Pick the method by where the answer lives: in what people say, what they do, or how that changes over weeks.
Which methods are evaluative research methods?
Evaluative research methods put a concept, prototype or live product in front of users to measure whether the solution works, as in usability testing, tree testing and closed card sorting.
The table compares six UX evaluative research methods by what they check, sample size and the kind of evaluation they serve.
| Method | What it checks | Sample size | Formative or summative |
|---|---|---|---|
| Usability testing | Whether users finish tasks on a prototype or product | 5 participants per round of qualitative usability testing (NN/g) | Formative; summative as a benchmark |
| Tree testing | Whether users find items in a text-only menu | 30–50 participants with the card sort (our program) | Formative |
| Closed card sorting | Whether users file content into set categories | 30–50 participants (our program) | Formative |
| Concept testing | Whether a design concept makes sense to users | A small sample | Formative |
| A/B testing | Which live version performs better on one metric | Enough traffic for statistical significance | Summative |
| Usability benchmarking | How a release performs against a prior version or a competitor | About 40 participants for a quantitative test (NN/g) | Summative |
A heuristic review, where experts check a design against usability principles, involves no users, so it complements evaluative research with users rather than replacing it. In our program, evaluative studies (4–8 days) use card sorts, tree tests and concept tests to check the early information architecture and design concepts. Teams that need several test rounds on one prototype can book our usability testing services as a separate engagement.
What are formative and summative evaluation?
Formative evaluation tests a design while it is still changing to learn what works and why, and summative evaluation measures how well a finished solution performs against a benchmark.
Kendrick describes formative studies as iterative, usually with a few users, and summative studies as a comparison against a benchmark, usually right before or right after a redesign. Evaluative research contains both, so it is not a synonym for summative research. Maze's guide to evaluative UX research adds a third type, outcome evaluation, which measures the changes a design brings, such as shifts in user behavior and satisfaction.
- Generative research: sibling mode, explores the problem.
- Evaluative research: tests a solution.
- Formative evaluation, during design: tree tests, closed card sorts, concept tests (our program), usability rounds.
- Summative evaluation, at or after launch: usability benchmarking, A/B tests.
- Outcome evaluation (Maze): changes in behavior and satisfaction.
Sources: Alita Kendrick (2019) and Christian Rohrer (2022), Nielsen Norman Group; Maze.
For where each type sits across a product's life, use the full stage-by-stage method map. Label each planned study formative or summative before you size it.
Can you give an example of evaluative research?
An illustrative example of evaluative research is testing the expense app's new receipt capture flow with a tree test and a moderated usability round before any code is written.
The team tree tests the new Expenses menu with 30–50 participants, then runs a 5-user moderated usability round on the prototype; success means a manager submits a receipt unaided. After launch, a summative follow-up compares on-time reports with the old flow. Write your own success criterion before your prototype is finished.
Which methods serve both generative and evaluative research?
Several methods switch between generative research and evaluative research because the research question, not the method, decides whether a session explores the problem or tests a solution.
Christian Rohrer's NN/g stage map lists surveys in both the generative strategize stage and the summative launch-and-assess stage, and it places concept testing in strategize. Our program runs concept tests in the evaluative studies step to check design concepts; NN/g files them earlier, in strategize, where methods "help generate ideas and answers about which way to go."
| Method | Generative when | Evaluative when |
|---|---|---|
| User interviews | Open questions about current work | A concept is shown and probed |
| Surveys | Open questions about needs | Customer satisfaction (CSAT) or Net Promoter Score (NPS) after launch |
| Card sorting | Open sort: users create groups | Closed sort: users file into set groups |
| Concept testing | An early idea that probes the problem (NN/g) | A design concept in our evaluative studies step |
| Diary studies | Routines before a product exists | Living with a beta release |
| Analytics | Points interviews to where users drop off | Measures a shipped change |
Fixed at the ends of the dial: contextual inquiry and participatory design (generative), tree testing and A/B testing (evaluative).
Our program runs its card sorts in the evaluative studies step, which follows generative fieldwork and checks the early information architecture. Write the research question before choosing the method: if it contains a design, the session is evaluative even when the calendar calls it an interview.
How do generative and evaluative research combine?
Generative and evaluative research combine in a loop: generative research frames the problem, evaluative research tests each solution against it, and every failed test sends the team back a step.
In practice, the evaluative vs generative research order matters less than the hand-off between them, so the choice is rarely either-or. In the Design Council's Double Diamond, Discover helps a team understand, rather than assume, what the problem is (generative), Define reframes the challenge, Develop generates different answers, and Deliver tests them at small scale (formative). Post-launch, summative evaluation and continuous research, through feedback surveys, product analytics and support channels, often raise the next generative question.
When an evaluative study fails, three checks tell you which way to go back.
- Users understood the goal but failed the steps: a design problem, so iterate in the formative loop.
- Users finished the task but would never do it this way, or at all: a problem-definition gap, so go back to generative research.
- Failures cluster in a user group nobody interviewed: recruit that group for generative work before the next test.
Behind the loop: Discover (generative), Define (synthesis), Develop and Deliver (formative testing).
On the expense example, a manager who cannot find the capture button points to check 1; one who says an assistant files every receipt points to check 2.
Our case studies show each half. On the Mass Media Co. platform revamp, user interviews were in scope, and we ran a discovery workshop with stakeholders and users to understand the business goals. The TradeZella case study shows the evaluative half: its visuals were tested with traders before launch. Before your next test, write down which check would send you back.
How long do generative and evaluative research take?
In our program, generative research takes 7–15 days of generative fieldwork to explore the problem, and evaluative research takes 4–8 days of evaluative studies to test a solution.
The fieldwork duration runs longer because each user group needs its own interviews, contextual inquiry means on-site visits, and open notes take longer to synthesize. The study duration is shorter: participant recruitment is an earlier step of its own, and the card sorts and tree tests run as online Optimal Workshop studies; run on its own, usability testing takes 1–3 weeks per round.
The program length is 4–8 weeks (20–40 working days) for one pass through the loop, from research framing to synthesis and readout, and the program price sits in our pricing section, which shows what a program with both modes costs. Recruiting several user groups while designers keep prototyping is the slow part, so we screen and schedule participants in participant recruitment (3–6 days), before any session. List your user groups now, because each one adds recruiting days before the first session can start.
What goes wrong in generative vs evaluative research?
Mistakes in generative vs evaluative research come from picking the wrong mode: testing a prototype to find what to build, or exploring the problem when a design only needs checking.
- Closing a usability test with demand questions. A participant who just finished a task rates the prototype, not their need, so collect demand evidence in interviews about current behavior.
- Pitching the concept in a discovery interview with users. It turns exploration into validation, so save concept probes for the end.
- Treating a passed usability test as proof of demand. Formative success shows usability, not that the problem matters.
- Reading a 5-user round as a benchmark. A formative round shows which problems exist, while a summative benchmark needs about 40 participants, so plan it as its own study.
- Skipping generative studies because analytics exist. A drop-off chart locates a problem in the funnel but cannot explain it, so pair it with interviews.
- Swapping users for a heuristic review. Experts predict usability problems but cannot reveal needs.
Check your next study plan against these six before you book anyone.
Who should run generative and evaluative research?
One research team should run both sides of the generative vs evaluative research split when an unstudied problem and an untested solution must be settled within the same 4–8 weeks.
Outside help makes sense when:
- No researcher is free for fieldwork while designers prototype, so the two modes would queue behind each other.
- A failed test needs a fast diagnosis, and the team holding the interview notes can tell a design failure (check 1) from a problem-definition gap (check 2) without a second hand-off.
- Stakeholders want findings from people who did not design the prototype, so results do not read as designers grading their own work.
Our generative and evaluative UX research program is built for that case.
Run the verb test on your next two product bets, one to explore and one to test. The bet with no screen behind it needs generative research first; the one with a prototype needs an evaluative round, or a short generative round first if nobody knows whether it solves the right problem, and the diagnosis loop above tells you where to go if a test fails. Settled on paper, the generative vs evaluative research question costs a sentence instead of a study.
Send us your next two product bets if either has nobody free to run it, and we will say which needs generative fieldwork first and which only needs an evaluative round. Every project begins with a free consultation and a fixed-scope proposal.
FAQs
What are the two main types of user research?
Split by goal, they are generative research, which explores the problem, and evaluative research, which tests a solution. Split by data, they are qualitative and quantitative research; market research, by contrast, sizes demand rather than studying how people use products.
Is usability testing generative or evaluative?
Usability testing is evaluative, because it needs a design to test. On a prototype it is formative evaluation, and as a benchmark on a live product it is summative evaluation.
Can one study be both generative and evaluative?
Yes, if the generative part comes first. An illustrative split is 40 minutes of open questions about current work, then 15 minutes of reactions to a concept, because a concept shown first colors the answers that follow it.
How many participants do generative and evaluative research need?
Generative research needs 10–15 interview participants per user group in our program, plus 4–6 users for contextual inquiry. Evaluative research needs 5 participants per round of qualitative usability testing, per Nielsen Norman Group, 30–50 participants for a card sort and its tree test in our program, and about 40 participants for a quantitative test.
What are the three main types of research in UX?
In NN/g's product-stage view, they are generative, formative and summative research, matched to the strategize, design, and launch-and-assess stages. Formative and summative research make up evaluative research. The User Interviews field guide names generative, evaluative and continuous research after launch, so adding continuous research to NN/g's three makes four.
Is generative research qualitative?
Yes, mostly. Interviews, contextual inquiry and diary studies produce qualitative data, but a survey can be generative when it explores the problem, and Rohrer lists surveys in the strategize stage as well as in launch and assess.
