How to Design a Meaningful Codebook in 6 Easy Steps

Customer Experience

Codebooks (or code concepts) are necessary to organize meaningful insights from a large set of open ends. Asking open questions instead of multiple choice questions with limited answers will increase your chance to capture more authentic and surprising answers from your target group. Responses can heavily vary in their expression and form, even when portraying a common theme. A codebook is a collection of topics, where topics might have one or more sub-topics – or codes. In order to make sense of it, you’ll need to choose a set of topics, that is comprehensive enough to address multiple types of survey responses and narrow enough to generate actionable feedback. Creating this balance is a challenging task.

These six steps will boost your ability to create meaningful codebooks:

1. Put yourself in the shoes of your customer
2. Determine key phrases and keywords
3. Create codes by “spreading the net wide” – and then, consolidating
4. Differentiate only what can be told apart concisely.
5. Categorize your codes and name them well.
6. Don’t force it

Even though humans excel at empathizing, they lack the ability to comb through a vast amount of data, find important keyphrases and interpret them at scale. Using an AI-assisted approach to create a codebook and analyze your data will not only boost your insight quality and speed but will also have a huge effect on your efficiency.

Use Open-Ends and an Organised Codebook

Collecting open-ended feedback from your customers is invaluable to your business. Rather than collecting “stock answers,” you get authentic responses from customers who are interested in sharing their thoughts, opinions, and actionable insights with you. As a result, you’ll be able to make informed decisions about product/service changes and significantly improve customer satisfaction. But in order to effectively collect – and assess – open-ended feedback, you’ll need a meaningful, productive codebook.

What is a Codebook, Anyway?

Open-ended feedback is notoriously challenging to assess. In an ideal world, all your customers will use the same language to describe similar complaints or insights. But in reality, that’s just not the case. Some of your customers might say your product is “too expensive.” Others will say “your price is too high.” Still, others will say, “I can’t afford this.”

That’s where a codebook comes in: It captures these different terms under one umbrella term. A codebook is basically a list of topics used for organizing verbatim answers. For example, all of the responses described above would be funneled into the code “Pricing: Expensive.” Instead of sifting through the answers one by one, you get instant analysis of open-ended feedback…and instant insight.

The Challenge: Creating an Effective Codebook

Creating a meaningful codebook, that suits your business needs, is a challenge. You’ll need to choose a set of topics or codes, that are comprehensive enough to address multiple types of survey responses and narrow enough to generate actionable feedback. You’ll need to think through what kinds of topics will “help” artificial intelligence funnel the answers correctly. And lastly, you’ll need to consider how to phrase the names of topics.

Here are six steps to help you create a meaningful codebook:

#1: Put yourself in the shoes of your customer.

Putting yourself in the shoes of your customers can be challenging for any business owner. You know every aspect of your product or service. You’ve invested time, money, and resources into development, strategic pricing, and functionality. It’s hard to imagine anyone would have anything to complain about!

Still, you’re guaranteed to have a number of customers who have suggestions, critical feedback, and even complaints. And missing out on those insights could be detrimental – even fatal – to the success of your business.

To put yourself in the place of your customer, you’ll need to distance yourself from your product or service. Take a step back and try to imagine you know nothing about your product or service. If it helps, ask (honest) friends or family members to give their opinions. What kinds of answers might your product or service generate to an open-ended feedback question? This line of thinking will help you to get started on creating a list of topics that are both relevant and realistic.

#2: Determine keyphrases and keywords.

Keyphrases are not the same thing as codes. Codes function as broad topics; keyphrases are narrower in scope. For example, a cell phone provider might use the code
Customer Service: friendly. Keyphrases that would fall under this code might include Lovely customer support, Nice agent, and Love the friendly staff. Keywords might include “warm,” “personable,” and “nice.”

In the example below, you’ll see the code Pricing: Expensive on the right and the original verbatims on the left. This example illustrates the purpose of codes very well: Grouping a set of quite varied phrases, including misspellings, into consistent groups.

Of course, you can generate your own key phrases and keywords, but you’ll also want to use a form of artificial intelligence to predict words and phrases your customers might use in a survey.

#3: Start by “spreading the net wide” – and then, consolidating.

Determining keyphrases and keywords will help you to put together a list of likely codes for categorization. At the beginning of this process, “spread the net wide.” Take a look at the codes generated by AI, and come up with your own. After sifting through about 50 verbatim phrases, begin to consolidate and reduce your codes. Resist the temptation to create a new code for every potential feedback. Ultimately, having too many codes will damage the depth of your insight and analysis. For average studies, we usually suggest ending up with somewhere between 20 and 60 codes.

#4: Differentiate only what can be told apart concisely.

You might create one code that senses a number of verbatim responses. But after sifting through some more responses, you might realize that this code isn’t as well-defined as you first imagined. If this uncertainty strikes repeatedly, consider merging this code with another to give you more consistent results. For example, you might merge the codes Customer Service: Long wait time and Customer Service: Make more responsive. Both of these phrases will be used to communicate essentially the same idea. Don’t torture yourself by differentiating between codes that are hardly distinguishable.

#5: Categorize your codes and name them well.

Codes function as categories in and of themselves, but they should also be grouped into higher-level categories. Your upper categories might include “Pricing,” “Customer Support,” or “Usability.”

Names should be concrete and concise. A bad example of a code name would be, Location has satisfying vibe / makes you like spending time there. A better name would be Atmosphere: Positive.

In any case, don’t include more than 10 codes in one category.

#6: Don’t force it.

Around 10% of your responses from a customer survey won’t be helpful to you. Either they are nonsensical (“asdf….”), useless (“I don’t want to write anything”), or too generic to be helpful (“great job.”) Don’t strive too hard to interpret an open-ended response that doesn’t serve your business, or use it to generate a new code. Ultimately, that approach will damage the effectiveness of your analysis. Instead, create one catch-all code that will be useful for the less-than-useful responses, i.e. “Not Applicable.”

Likewise, consider actionable results.

Be pragmatic.

An NPS Survey: Your Best Bet For Generating High-Value Feedback

Ultimately creating a codebook can be an extremely valuable strategy for turning your open-ended feedback into valuable, actionable insight for your business.

Of course, you might be thinking, Sounds great! But how do I generate open-ended feedback? 🤔

Gathering open-ended feedback for your business is most effectively done with a Net Promoter Score (NPS) survey. This simple, 2-question survey allows you to gather NPS – a metric used to generate customer satisfaction – and open-ended feedback, asking customers to explain their ratings.

Caplena: Helping You Design a Meaningful Codebook for Optimal Results

Caplena is an AI-powered software that can significantly simplify the process of feedback analysis and codebook design – allowing you more time to work on your product or service, reach new clientele, and strategize for your business.

How can we help?

• Generate potential keywords and keyphrases for a meaningful codebook.
• Automatically code open-ended texts.
• Analyze different sentiments using NLP (Natural Language Processing).

Caplena also offers you codebook templates that make the process even easier.

Other Useful Links:

5 Steps to Go from a Notebook to a Deployed Model