Open text questions can be a great addition to your survey. They allow respondents to answer in open-text format, giving them the freedom and space to answer in as much (or as little) detail as they like. You can use them to allow respondents to elaborate or expand on their closed answers or to ensure that you are not limiting your responses to a certain question to a fixed number of options.
The analysis of open text questions can also be more resource intensive. Translating the many responses into actionable data and incorporating them into reports can take some skill and time.
“Beta” automated text analysis
Great news! Analysing open-ended responses has just become a whole lot easier.
We’ve partnered up with our data science team to build a first version of our automated text analysis tool. This feature automatically scans all your open-ended responses, detracts important keywords and phrases then groups them together, making it easier to discover insights without needing to plough through all of your individual responses.
You’ll be able to access a list of all your keywords that have been recognized by our model, with the count, frequency in %. And you can see your results visualised in a word cloud.
We’re still working to improve this feature, but we wanted to give you access to the beta feature so you can already benefit from the automated grouping. As we want to continue improving our text analysis tool, we’re looking for your feedback! If you have any thoughts to share (what you like, what you dislike, what you're confused by) please contact your CSM or let us know via the in-platform live chat.
How does it work?
When your results are in, you can navigate to the "trends" page on the left hand side, on the "overview tab" you will see an overview of all your questions and responses. On your open ended questions, you’ll now see a link to “view text analysis”.
When you click on this link you’ll be taken to the text analysis page of your survey. It can take up to 1 minute for the analysis to be run, but this will only be the first time you access this page.
When the text analysis has been completed, you’ll see a list of all the keywords and phrases that the model has recognized, including a count and the % frequency, which is based on your total number of responses.
You can also see your keywords and key phrases visualised as a word cloud. Word clouds are a great way to visualize text data and get an overview view of keywords and trends. A word cloud shows a cluster of words, with the size of the word indicating how often this word has been mentioned in your responses.
You can choose the amount of keywords visualised in your word cloud (ranging from 1-50) by filling in the input field to the bottom-right of your word cloud. You can also remove certain keywords from your word cloud. To do this you’ll need to hover over the keyword in your list on the left and click on the the eye icon to hide this word from your word cloud.
To see the individual responses that include each keyword, you can click on the arrow next to the keyword. This will show all the original responses that contain this keyword. It’s possible that a single response can have multiple keywords. In the example below the answer “Sunshine, Cocktails, Relaxing” contains three keywords (”Sunshine”, “Cocktails” and “Relaxing”).
If you wish to change between markets, dates or questions, you can use the panel of the right. You can collapse this panel by clicking on the arrow, to make more space for your word cloud. Keep in mind that you will only be able to select from questions, dates and countries where text analysis is available (all English surveys, send on or after 1/1/2021).
If you want to export your text analysis you can choose to “copy results”, this allows you to paste the list of keywords, count and frequency percentage into a spreadsheet, or you can directly download your word cloud as an image.
At the moment automated text analysis is only available for English surveys, run on or after 1/1/2021. The model is run on all your responses, so it will not take into account any filters that you have applied on your dashboard.
How to edit your text analysis
You can choose the amount of keywords to be included in your word cloud (ranging from 1-50). You can also exclude certain key words from the word cloud by hovering over them in the list on the left and clicking on the eye icon.
At the moment you can not remove, merge or edit any keywords or phrases yet, but we’re working on adding this functionality soon. In case you do want to make certain changes, you can choose to “copy results”. This will copy all your keywords, the count and the frequency to your clipboard, allowing you to easily paste this into a spreadsheet.
What is the algorithm behind it?
We’ve spent months building and training our own data model for this feature, based on the millions of open-ended responses that we’ve gathered over the years.
There are two important steps in the way the model works:
Key phrase extraction
Key phrase grouping
The key phrase extraction algorithm is a deep-learning model that has been trained to extract important/relevant keywords in open text.
The key phrase grouping algorithm takes these key phrases and then groups similar key phrases into the same group. What’s unique about our key phrase grouping is that it isn’t just based on a simple text match, but based on deep-learning. This means that it’s able to also recognize terms that are semantically similar (for example, ‘amazing’ and ‘wonderful’).
We’re still aiming to further improve this model, so keep in mind that while it’s built on a solid foundation of data science, certain words might not be picked up by our AI model as a keyword. Also, while our model implicitly understands that words that are spelled similarly (i.e typos,
dog) have a high similarity, this doesn’t mean that the model will be able to pick up on any type of spelling mistake.
If you feel that there’s a significant different between the automated text analysis and your original responses, it would be great to receive this feedback. We are always looking at how we can improve our data model and this might be useful information. Reach out to your CSM or contact us via intercom.