You’re well aware that there’s much, much more to surveying than simply writing down some questions and emailing them to your users. In fact, the questions themselves are just one of the three complementary components that make up the survey as a whole—no one part is more important than the others, and without attending to all three your survey efforts will not achieve maximum success. This three-part framework consists of the who, the how, and the what…
Who – Whether it’s your website visitors, a panel of strangers, or your established mailing list, who you ask is integral to the success of your survey. We will cover this element in Chapter 5.
How – This refers to the type of survey you choose. You could certainly try using an email-based survey to get to the root of your landing page’s high bounce rate, but you probably wouldn’t have much luck. We touched on survey types in Chapter 1, but we will discuss this element at length in Chapter 6.
What – This is what most people think of when they hear the word survey—the actual questions. If you ask the wrong questions to a meticulously selected demographic using the perfect survey method—you’re still asking bad questions. We’ll talk about this element in Chapter 7.
Before we dissect these individual parts, however, it’s important to look at the whole. In this chapter we’ll analyze the survey process from start to finish.
1. Establish Main Objective (Why) and Target Audience (Who)
This could be treated as two separate parts, but since the latter is so dependent on the former, many marketers choose to consider them one. It is unwise to begin surveying without first isolating the why and the who (as we’ll discuss at length in Chapter 5). One of the most common mistakes is surveying for survey’s sake without any aim or direction. To avoid this pitfall, it’s important to identify your main objective and your target audience before doing anything else.
2. Build the Survey
At most, a survey consists of three parts: the introduction, the questions themselves, and the conclusion. We’ve outlined each one below.
- Introduction (Optional, Depending on Survey Type) – While page-based surveys tend to have context built in and thus need no introduction, custom, user satisfaction, and product-market fit surveys tend to require some additional context and explanation. Depending on the situation, the introduction might include:
- A short summary of the survey’s purpose or objective
- A confidentiality agreement
- How many questions the respondent will be asked to answer, or how long the survey takes on average
- Ways to contact staff if any issues arise
- Mention of any incentives to drive participation for longer surveys
If driving total participation is important, strongly consider rewarding your participants in some way. If you’re able to offer a discount code, free download, or some other gift, then great. If not, make it clear that their time will be well spent making the product work better for them. This is enough for many users, especially if you aren’t asking them to answer pages upon pages of questions.
Example introduction:
Welcome to (Company Name)’s site usability survey.
Thank you for participating in our survey. We at (Company Name) value your feedback as we take steps to improve our site for you.
This survey should take no more than five minutes of your time to complete. Your participation in the survey will be completely anonymous. As a thank you for your participation, your email will be entered into a weekly drawing for a $100.00 gift card.
If you at any time run into an error or have any questions, please don’t hesitate to contact us at yourcompany@yourcompany.com.
- The Questions - After the introduction comes the actual survey questions. In Chapter 7 we’ll analyze the two basic question types as well as the actual writing of the questions, but here are a few tips to keep in mind…
- Ask for only need-to-know information—this is actionable data directly related to the survey purpose you established in Step 1. The nice-to-know stuff is great, but asking too much is the quickest way to overwhelm or scare away users.
- Break longer surveys into sections and include a progress bar so that respondents can see they are moving forward.
- Group similar questions together. To jump from subject to subject can be jarring for users, another potential cause of abandonment.
- Begin with broad, easy-to-answer questions, and save more sensitive and specific questions for the end of the survey. Building trust and momentum makes it more likely that respondents will answer the higher-friction questions toward the end.
- While brand consistency is important, be sure you aren’t putting undue hardship on participants by avoiding small or elaborate fonts and steering clear of clashing or hard-to-read color schemes.
- Bold your instructions and questions, but not the answer choices. You want to make sure users aren’t just skipping over questions. Making the questions stand out visually may make them easier to identify and read.
- Be consistent. Even small changes are noticeable and can erode trust. A jump from Arial to Times New Roman, for example, might make you seem sloppy, erratic, or, worse, untrustworthy.
- Randomize answer choices to discourage the response order effect, in which results are skewed because the top answers in a vertical list tend to be chosen more often.
- Thank You Page (The Conclusion) - Despite the importance of keeping it short and simple, a quick thank you tends to be a good idea. It doesn’t have to take up more than a single line, and it goes a long way. Something as simple as “Thank you for taking the time to fill out this survey” gets the job done. This not only clears up any confusion regarding whether the survey is actually over—a more common problem than you might think—but it also lets users know that you appreciate their time. No one will be offended by the extra ten seconds it takes to read a “Thank You” line, though some might be if they feel like you aren’t grateful.
3. Pre-Testing
Though this tends to be the most overlooked step in all of the survey process, it’s critical to pre-test and review both the survey copy and the system you are using to conduct it, otherwise you can’t be sure the feedback you’re getting is reliable.
This is especially true for surveys with branching (sometimes referred to as ‘skip’) logic—where users are presented different questions based on previous answers. This complexity can easily create errors in the survey flow that are only caught by rigorous testing.
Enlist a few people unfamiliar with the survey design and objectives and have them read the questions for clarity and comprehension. What do they think the questions are asking? Is this what you intended? Check the results they report against the results your survey software reports.
People who run into issues with particular questions, especially those required to complete the survey, may provide false information just to get past your survey error. This essentially ruins the data from that question, costing you valuable insights.
4. Data Collection, Reduction and Analysis
After you’ve ruled out any unexpected reporting errors, data collection can begin. We’ll discuss analysis and reduction further in Chapter 8, but here’s what you need to know for now:
This is where user surveys come in.
Though online surveying is just one of many ways you can collect feedback from users, we’ll be spending most of our time dealing with this form of surveys. That’s not to say that other methods don’t have their merits, but for online businesses looking to collect feedback regularly and at scale, online is a cost-effective and efficient way to do so.
- A lot will depend on your response rate. The more users you can get to actually complete a survey, the more accurate your results.
- Similarly, the purpose of your research will determine how you analyze the data. You may choose to count the frequency of responses, cross-tabulate responses, measure the change in responses to a question over time, or look at the numbers in some other way that makes sense for your research goals.
- Whether you plan to publish your survey results on your site or distribute them to investors or team members, remember to be objective in your analysis and to keep sight of your survey goals.
5. Communication With Respondents (Optional)
Not “optional” as in only if you feel like it, but optional as in not all survey types require you to touch base post-survey. If you implement an email-based survey or collect an email address as part of the survey process with a promise to follow up, this step is essential.
Having a communication plan in place for dealing with survey responses that illuminate severe issues or brand advocates is a good way to ensure that unique customer feedback is not only heard, but followed up with individually. Customers reporting a very poor experience should receive follow up outside of the survey process to try to repair the relationship, while exceptionally satisfied users might be identified for soliciting a testimonial or participation in a customer-focused ad campaign.
6. Implement Learning From Your Results
This is typically where the survey process breaks down—a critical survey mistake. Internalizing and acting on your feedback is definitely not optional if you want to keep improving your business. Learn from the feedback you accumulate, then use it to make your product or service and your future surveys better. Unless you’re tracking satisfaction over time, you shouldn’t be sending users the exact same survey every six months—unless you want to make them think you’re not doing anything with the feedback they’ve already given you.
Though it’s likely every step won’t apply to your survey situation, it’s helpful to observe the process in it’s entirety from the outside. Now that we’ve gotten that overview out of the way, the rest of the guide will break down and analyze each of the above steps in depth.
