Filter by Imported demographic data
Analyzing by Survey Question Type
Single select, Dropdown, and Multiple select
The Survey Analysis tool in the Results Dashboard is where you can explore responses to the closed-ended questions you asked your participants. Responses can be filtered by any of the response options you provided, by participation group, or by contact data using the Filters button at the top of the page. Leaders can also choose to include partial responses (data from participants who did not complete all the questions) or filter them out.
The Survey Analysis tab includes a number of sections, depending on the settings of your engagement: Languages, Participation groups, Survey responses and Compare. Languages displays the representation of languages used by participants, Participation groups shows the number of participants in each group, Survey responses provides a straightforward breakdown of your participants' answer to survey questions and Compare lets you map those responses onto one another across questions (more details below).
Filters
Engagements where survey questions or participation groups are used have a page-level filtering capability that allows you to see how different groups responded to each question..
Filter by Participation Groups
The participation groups filter is a single-select option which allows you to see the results from any individual group or from all participants in an engagement. . When you select a group and click “Apply filters”, each survey question will reset to display only the answers that came from that particular participation group.
Survey responses filter
You can also choose to filter out responses from participants who did not complete the entire engagement or include them in your analysis. If you want to see only the responses from participants who answered all of the survey questions, select “Only complete responses”.
The default selection is “Partial and complete responses”, meaning responses from all participants are included, regardless of whether they completed every question..
In a survey where participation is limited by unique links or code-authenticated access, only those participants who clicked the “Submit and complete” button will be included as ‘complete responses’.
Survey Questions
You can also filter your results using the responses to your survey questions themselves. This is a multi-select option where you can include all or a selection of answer groups from various survey questions.
Using the example below, if you want to only look at how parents of secondary school students responded, you can deselect the other options and click “Apply filters”.
Filter by imported demographic data
When an engagement is run with the confidential privacy setting and the engagement access is restricted to unique links or code-authenticated access, an additional filtering capability is available. Any demographic information contained in the contacts list that was uploaded to your account becomes available as a filter on the Survey Analysis page.
The filters are found under the title “Contact data” and multiple options can be selected at once.
Note: this feature is not available on all accounts. Reach out to your ThoughtExchange rep to learn more.
The Participant stats card
This feature lets you see how many people have started and completed the survey portion of an engagement.
An example where every participant who started the survey and completed it:
An example where only 50% of participants who started the survey completed it:
When the engagement utilizes unique links (email invitations), the "Invited" stats are also displayed:
Note: 'Responded' = number of invitees who answered at least one survey question
The completion % in the Participation stats card will include anyone who has completed the survey, regardless of whether they skipped any optional/skippable survey questions.
Within the Survey Responses tab, you will be able to see how many participants actually answered the question vs how many skipped.
Languages
If your engagement allows for participation in multiple languages, this section shows the different languages and the number of people in each of those groups.
You can filter this chart by both participation groups and survey responses in order to see the languages used by those subsets of your participants.
Participation Groups
If your engagement uses the Participation groups feature, this is where you can find the breakdown of each group.
Filters are not currently applicable to this tab.
Analyzing by Survey Question Type
Survey Responses displays the results summary for each of your Survey Questions. The way that data is presented is determined by the type of question asked (please note that not all question types may be available on your account. Contact your ThoughtExchange rep to learn more).
The “Answered / Not answered” stat
Each question will display two additional numbers: ‘Answered’ and ‘Not answered’.
‘Answered’ indicates the number of participants who responded to that question.
’Not answered’ indicates the number of participants who did not complete that question. This could be because the participant dropped off from the engagement, because custom survey question logic dictated that the participant wasn’t shown that question (due to how they answered a previous question), or simply because of question skipping.
Note: More niche cases for the ’Not answered’ stat could include:
- The question was added after the engagement was launched and already had responses; participants who had previously responded (before the question was created) would contribute to the ’Not answered’ stat.
- The leader deletes a specific answer choice that participants had selected.
Single select, Dropdown, and Multiple select
Single select, Dropdown, and Multiple select style questions are always presented using a bar chart with “Other (specify)” options listed at the bottom. Click any of the groups to see the thoughts they rated highest and lowest.
Ranking
The responses to the ranking questions type show as a card. A Ranking question card shows a list of answer choices. Each list row contains:
- Title (what the choice is about).
- The average ranking it got from everyone who answered the question.
- The average ranking is rounded to the second decimal place (e.g. 2.75, 4.68, etc.) and is rounded up or down depending on the value of the 3rd decimal (e.g. 2.754→2.75, 4.676→4.68)
- The overall ranking, which arranges all choices based on their average rankings
If two or more choices have the same average ranking, they receive the same overall ranking. In such cases, the next rank is skipped to keep each rank remains unique, even when choices are equally effective.
Likert-scale
Likert-scale responses are presented in a bar chart by default, but you can switch to a pie chart, if you prefer. Click any of the groups to see the thoughts they rated highest and lowest.
If you would like to combine any of the response groups to get an overall favorability score, use the drop down menu on the right-hand side of the page to select the relevant groups. Learn more by clicking the heading below.
Using the Likert Favorability Score
Here are a couple of key reasons why tying a favorability score to Likert scale question can be beneficial:
- Quantity results: By assigning a favorability score to Likert scale question results, you introduce a quantitative measure to evaluate responses. This allows for a more precise and objective assessment of the overall sentiment or opinion expressed by the respondents. Instead of relying solely on qualitative interpretations, you can quantify and compare the level of favorability across different questions or survey participants.
- Standardized Comparison: When Likert scale responses are converted into favorability scores, it becomes easier to compare and contrast results across demographics (department, school, role, gender, ethnicity, etc.) or survey iterations. By aggregating and analyzing the favorability scores, you can identify trends, patterns, and areas of strength or weaknesses more effectively.
- Simplified Visualization: Representing favorability scores derived from Likert scale responses allows for intuitive and concise visualization of data. While a bar or pie chart can help show a breakdown of responses, they fail to provide an easy answer to the main reason why the likert scale question is being asked in the first place (“how well are we doing on this topic?”)
- Goal Setting: This enables you to evaluate current performance against past results or a certain future target/goal. Tracking these scores becomes a valuable tool for monitoring progress, identifying areas for enhancement, and measuring the effectiveness of interventions or initiatives.
- Decision Support: Favorability scores derived from Likert scale responses can serve as decision-support tools. When making data-driven decisions, having a quantitative metric allows you to weigh the level of favorability for different aspects of a product, service, or experience. By considering favorability scores alongside other relevant factors, you can make more informed choices that align with the preferences and satisfaction levels of your target audience.
Overall, tying a favorability score to Likert scale question results provides a structured, measurable, and interpretable approach to capturing and analyzing respondent sentiment. It enables objective comparisons, simplifies visualization, supports benchmarking and goal setting, and enhances decision-making processes.
Satisfaction Score
Satisfaction Score questions provide an automatically-calculated score along with a bar chart breakdown of responses. The satisfaction score is based on an industry-standard metric named NPS (Net Promoter Score).
Participants answering this question will fall into one of three categories:
- Satisfied (9 or 10)– Typically loyal and enthusiastic participants willing to promote us.
- Passives (7 or 8)– Participants that are somewhat content with your service but not enough to be Satisfied.
- Dissatisfied (0-6)– Participants who have had negative experiences. These participants are at risk of churning/quitting, leaving negative online reviews, etc.
From there, we calculate the score by
- Getting the [% of participants that are Satisfied]
- Getting the [% of participants that are Dissatisfied]
- Calculating [%...Satisfied] - [#...Dissatisfied]
Use this score to measure how well you are doing. Use it over time to measure if you are getting better or worse.
This score is great for understanding how well you are doing, but it doesn’t really tell you what you can do to improve. Use a combination of Likert scale questions and thought analysis to better understand why participants feel a certain way about various themes influencing satisfaction (for EDU, questions can be about class sizes, mental health, curriculum, etc.).
Say you want to better understand the views of Dissatisfied participants. Simply Filter results by participants that answered [0,1,2,3,4,5,6] to a Satisfaction Score question to see how they answered likert scale questions, and how they rated thoughts.
Comment Box and Deep Dive
Responses from both comment box and deep dive type questions can be viewed either as a wordcloud of the most commonly used words in participants' responses or using the full list view (which shows each response in full). Toggle between these options using the buttons to the right of the question.
Wordclouds can be edited to remove unwanted words. Simply click the Edit button in the top right corner of the wordcloud and click any words you would like to remove. When you are finished editing, click the Done button below and to the right of the wordcloud. You can view the thoughts that used each keyword by clicking the relevant word (provided the Edit function is not active).
Note: Advisor and Article are two AI-powered features within the Results dashboard that provide you with user-friendly ways to analyze data and gain a more comprehensive understanding of the results from your comment box and deep dive questions.
Survey Question Matrix
When analyzing results, matrix questions are grouped together in a stacked bar format:
And leaders can take a closer look at each question (and define favorability scores, if applicable) by clicking any of the questions or bars:
Compare
The compare tool allows you to look at a variety of options: compare two survey questions, satisfaction scores, favorability scores, or demographic group sizes. These options are only visible if there are survey questions of those types to compare.
Note: comparing satisfaction scores, favorability scores, or demographic groups sizes requires the use of the Advanced Survey Question Comparison Tool, which is in beta testing. Click here to learn more.
Compare Two Survey Questions
Simply select the two questions you would like to compare and a table will automatically populate showing the count and percentage of participants who fell into each possible combination of responses. Use the Heatmap option in the upper right corner to layer on colors to help you quickly see the relative sizes of each group. You can even use the Filters button in combination with the Compare tool to look at more than two questions at once.
Note: Groups with fewer than 5 participants will not display to protect anonymity.
Compare Satisfaction Scores
When you select ‘Satisfaction scores’ from the ‘I want to compare…’ dropdown, you will need to click ‘Compare now’ to be taken to the Advanced survey question comparison tools.
Note: your engagement will need at least one satisfaction score question and one demographic type question for this compare tool to work.
Compare Favorability Scores
When you select ‘Favorability scores’ from the ‘I want to compare…’ dropdown, you will need to click ‘Compare now’ to be taken to the Advanced survey question comparison tools.
Note: your engagement will need at least one Likert scale question and one demographic type question for this compare tool to work.
This section also allows both for comparing favorability scores at the topic level (learn more about topics here) and for comparing across two engagements.
Compare demographic group sizes
This feature displays the demographic group sizes in one easy-to-read table.
A note about the ‘Advanced’ options
There are some extra options you may wish to include in your analysis, depending on what you are trying to accomplish.
“Show answers that have 0 responses” means that answer options that weren’t selected by any participants are still included in your charts or table. Deselect this option if you would like to exclude the answer options that no one selected.
“Include partial responses” means that all answers are included in the charts and tables, even those from participants who dropped off before answering all questions. Deselect this option if you would like to exclude the answers from these participants.
“Show count of likert scale responses by demographic group” displays the count of participants who responded with each option to your demographic questions. This option is deselected by default.