Designing a great survey isn’t about cramming in clever questions. It’s about engineering decisions to lead to better outcomes. The goal is to collect valid, reliable, actionable data with minimal burden on respondents.
1) Start with the decision–Not the question
Why it matters: Questionnaires should be aligned to the decisions you’ll make. Start by asking, “What will you do differently with the results?” This will help you translate decisions to outcomes.
Then, narrow your focus to make sure your questions fall into the categories below:
- Descriptive → “What’s happening?” (understand current state)
- Diagnostic → “Why is it happening?” (underlying drivers)
- Predictive → “What will happen?” (future impacts)
Example:
- Outcome: “Improve family/caregiver communication this term.”
- Topics: timeliness, clarity, channel effectiveness.
2) Ask item-specific questions (avoid agree/disagree)
Why it matters: Agree/Disagree formats invite acquiescence bias (a tendency to agree), especially among less experienced respondents. Use item-specific, forced-choice wording instead (Pew Research)
Example (before/after):
- Weak: “I feel informed about school policies.”
Better: “How informed are you about school policies?” (Not at all → Very)
3) One idea per question (kill double-barrels)
Why it matters: Double-barrelled items (“timely and clear”) can’t be interpreted. Split them. Research on the cognitive steps of answering shows that small wording changes alter responses (AAPOR)
Example:
- Weak: “Communication is timely and clear.”
- Better: “How timely is school communication?” and “How clear is school communication?”
4) Control order effects & optimize flow
Why it matters: Order and context shape answers. Go general → specific → sensitive; group by topic; front-load important content (Cambridge University Press & Assessment)
Example flow:
1) Demographic→ 2) Overall satisfaction → 3) Communication items → 4) Support items → 5) Open-end(s)
5) Minimize cognitive load
Why it matters: Short, plain-language items reduce confusion and speed completion; huge grids increase break-offs. (AAPOR)
Example:
- Replace “SEL” with “social-emotional learning (SEL)” on first mention.
- Break large matrices into 2–3 shorter pages or single items.
6) Use branching where it truly adds value
Why it matters: Targeted follow-ups keep surveys short and relevant and reflect the “tailored design” approach to respondent experience.
Example (conditional):
- If “Dissatisfied” with communication → “What is the main reason?” (open) → “Which fix would help most?” (single-select list)
7) Mix closed and open-ended items (sparingly)
Why it matters: Closed items power charts; purposeful open-ends explain the why and capture solutions. Place open-ends after a scale item for context. (AAPOR)
Example:
- Likert Scale: “How easy is it to get help with tech issues?”
- Open-end: “What’s one change that would make support easier?”
8) Pilot before launch (cognitive pretesting)
Why it matters: Cognitive interviews catch misinterpretations before they’re widespread. Time the survey (≤7–10 min for most audiences).
Example protocol:
- Ask 5–10 people: “Tell me, in your own words, what this question is asking.” Revise items they stumble on.
9) Sampling & outreach plan
Why it matters: Getting the right voices and enough of them depends on who you invite and how you contact them (email, QR, SMS) with planned reminders. (Tailored Design Method)
Example:
- Invite Monday morning; reminders Wednesday and the following Monday; SMS nudge for groups with low email open rates.
Sources & Further Reading
- AAPOR. Best practices & question wording (bias, ethics, reliability). AAPOR+1
- Pew Research Center. Writing Survey Questions & Methods 101 (agree/disagree, item wording). Pew Research Center+1
- Tourangeau, Rips & Rasinski. The Psychology of Survey Response (order/context, cognition). Cambridge University Press & Assessment
- Dillman. Tailored/Total Design Method (contact strategy, progress cues, respondent experience). sesrc.wsu.edu+1