When you’re conducting online surveys, whether it’s for market research, customer satisfaction polls or other ends, there are a number of best practices that can help boost your response rate and produce more dependable findings. In this How-To, we outline what you need to keep in mind, from survey creation to dissemination to data collection — all with an eye on making analysis easy. Discover 10 best practices to creating useful surveys, the 3 types of questions to use, and 6 tactics for boosting your response rate.

10 Best Practices for Creating Effective Online Surveys

1. Know your objective. This seems obvious, yet few people take the time to write out a clear objective to any survey. The fact that surveys can be used to gather customer feedback, benchmark an industry, or research a potential market for a new product means that marketers sometimes confuse two objectives in one. A written statement can help guard against scope creep, and should be at the top of the document where you brainstorm/formulate your questions.

You should also have a fairly good idea of what you’re going to do with the responses you receive, i.e., are you going to make tables, bar charts or line graphs.

Keep in mind that while pie charts are nearly ubiquitous in market research, especially B2B research, the human eye has an easier time understanding comparative values when they’re in a bar graph vs. a pie chart.

2. Determine the Sample Size you need. If you plan on running any sort of statistical analysis on the data you receive, you’ll want a 95% – 99% confidence level with no more than 5% margin of error (i.e., confidence interval).  It’s best to establish what sort of confidence level you want, the size of the population you intend to sample and then determine the number of responses you’ll need for a representative sample. (Reversing the process by sampling a population and then determining how representative your sample is of the population is just messy and often leads to unrepresentative data).

You can use an online calculator to determine number of responses you’ll need, but here are some general benchmarks:

As you can see, you need a rather large percentage of a small population to produce generalizable results, but after 50,000, you only need between 380 and 390 respondents to have solid data.

Of course, if you’re not running significance tests or any statistics, you can work with smaller sample sizes. But your data will then be anecdotal, not generalizable, and should be interpreted with caution.

** Be advised that online surveys are particularly susceptible to non-response bias, that is, the bias that is introduced when respondents to a survey are different from those who did not respond in terms of demographics or attitudes. For example, younger doctors are more likely to respond to online surveys than older ones, but that also makes them less experienced as physicians, so a company looking to collect information on physician expertise will get biased results through an online survey. Similarly, your customers are often not representative of the larger market you serve.

Also, as a general rule of thumb, your survey response rate will be less than your open and clickthrough rates for email marketing, but you should get more than a 1% response rate. The tactics below will help you boost that number.

3. Write clear questions. That means clear syntax and not combining ideas that could be interpreted differently. So-called compound questions are hard to answer accurately and generate inaccurate responses. For example:

From which types of sites do you get your news and entertainment?

is better phrased as two, shorter questions:

Where do you get your news online?
What sites do you visit for entertainment?

At the same time, you may sometimes need to use longer phrases to convey clarity. For example,” Where do you get your news online?” can prompt responses from websites to social media to search engines. But if you only want to know about competing sites, you could ask:

Which websites do you visit for news?

To that end, you should also make sure the first question you ask is interesting/engaging and easy to answer (see #4 for tips on making questions easy to answer).

4. Use closed questions. Both your respondents and your analysts are pressed for time. Open-ended questions are cumbersome to answer and cumbersome to code and analyze. Keep things simple whenever possible by using multiple-choice or rating-scale questions. Of course, some questions need to be open-ended, but we recommend never including more than two open-ended questions (no matter how long your survey is) and they should be thoughtful, such as “What else would you like to tell us that we haven’t asked about?” Even the amount of space you leave for an open-ended question matters: leave too much space and many people will assume you want a detailed answer, and will often skip it as a result. Visual cues matter in surveys!

The 3 major types of closed questions are:

A. Multiple choice questions. It’s best to have no more than five options (and no less than two). Also make sure there are clear delineations between benchmarks.

In this example, people who are selling between metrics (i.e., 3 years and a few months, maybe) are forced to make a choice between three and four years, but by doing so, you get data that’s more easily benchmarked.

Also, keep in mind that if you plan on charting the percentage of respondents that clicked on answers in a multiple choice question, it helps if the answers aren’t percentages themselves. Stating or charting that “30% of respondents reported getting between 11% and 50% of website traffic from search engines” is confusing for readers. Sometimes this is inevitable, but try to avoid it when you can.

B. Radio-button or check-box questions. Again, keep the number of options low, between 2 and 10. If you expect that your respondents might have a response you’re not prepared for, you can add a fill-in “Other” option, but use this sparingly and only when necessary because it’s really a headache to code down the road.

C. Likert scale questions. Rating scales, known formally as Likert scales, can help categorize more qualitative answers. However, don’t use a 1-10 scale. People tend to answer on the opposite sides of a spectrum or the middle, making five the ideal scale breadth. Also, make sure the scales are consistent throughout the survey, and that the middle response is appropriate. For example, the scale below makes little sense since people will either recommend a website to someone or not — no one is actually neutral. A better category would be “Depending on circumstances”.

Also, don’t use always or never as choices; they tend to bias responses to the extremes. Or, they interpret “never” more literally than you’d imagine and never check it, which also muddies results.

5. Keep it short. Like we said, people are pressed for time. A study by software company Vovici found a direct correlation between the number of questions in a survey and abandonment rates:

If you use mostly closed questions, you can responses to around 40-50 questions in a 10-15 minute timespan. But you’ll be hard-pressed to get anyone to spend more than 5 minutes (and even that’s a stretch).

Also, don’t include a visitor counter on survey pages.

6. Account for all possible answers. While you should try to have the fewest options possible in order to boost response rates and decrease time spent on answering your survey, you also want to make sure that people aren’t forced to choose an answer when they really don’t know or the question isn’t applicable to them. So be sure to include “Not Applicable” or “I don’t know” whenever it’s appropriate.

7. Use logical ordering. Keep similar topics together, and follow a logical order. Start with screening questions (and thank those who don’t qualify), then move through the general, closed questions. It’s best to keep open-ended questions until the end (although sometimes it makes more logical sense to group them with questions on similar topics).

8. Use branching and skip logic. Some survey creation software, like Formstack, lets you create rules by which someone has to respond to a question to see another question. Most marketers use this “branching” for secondary and tertiary questions, but really, your whole survey form can be coded this way so that respondents only see one question at a time. Also, you should utilize skip logic so that if someone answers no to a qualifying question, they will not be asked the related “branch” questions, but will instead will skip down to the next section. Side benefit is fewer questions to answer overall, so improved response.

An alternative to branching is to separate questions on different pages in small bites (preferably without the need to scroll) and a progress bar up top. However, every “page turn” is an opportunity for people to abandon your survey. If your survey software doesn’t collect partial answers, this can seriously lower your response rate and data collection. Keeping respondents on the same page by revealing one question at a time is the best practice. If you’re forced to use multiple pages, then at the very least, use a progress bar so that respondents can budget the appropriate amount of time from the start.

9. Test before sending. Before you send out your survey, have a few people unfamiliar with it run through it and submit answers so that you can catch any glitches.  Also, make sure to test for different browsers, computers and devices.

10. Learn from past results. If a survey question provided no valuable insight, don’t ask that question again. If respondents left certain questions blank, either drop them or find a way to answer them that might yield a result (often, this involves turning an open-ended question into a closed one)You’re your survey really warrants it, you can sometimes ask a critical question at the beginning and then ask a rephrased version of the same question at the end for additional validity/insight into your population.

6 Tactics to Boost Response Rates

People will spend days to months constructing a survey and then dash it off with an email that simply reads “Please take our customer service survey.” But using your marketing skills to “market” your survey can often help boost response rates (just don’t try a bait-and-switch).

Here are 6 tactics to help you market your survey and boost your response rate:

1. Create compelling marketing copy for emails, social media and on-site reminders. What’s your hook? What’s the benefit to the respondent for taking time out of their busy day to answer you? If you can’t answer these questions, you’re going to have a hard time “selling” your survey to respondents.

To that end, your survey page should have a catchy headline that impresses and makes people want to stay on the page. “Market Research for XYZ Site” is a snooze-fest that doesn’t inspire participation.

Also, give an accurate time expectation in concrete terms. Say “this survey should take 5 minutes or less,” not “take this brief survey,” since “brief” can mean different things to different people.

And remember, the WIIFM (what’s in it for me?) doesn’t have to be overwrought. Simply saying you are asking a small group of people these important questions, and they’ll impact (for example) your new product development, can make them (and the survey) feel more important and significant.

2. Send on a good day. Recent research shows that the highest open and click through rates for survey responses happen on Monday, Friday and Sunday, so you might want to time your survey to go out then.  But feel free to use email testing and onsite analytics to see when your audience is most responsive. Also be wary of busy periods in your industry and in life in general. It’s generally a bad idea to send surveys out the week of Christmas or April 15th (tax day in the US).

3. Pre-populate forms. If you already have subscriber information (email, age, gender, job title), don’t ask for it again — just ask respondents to verify the information you have. HOWEVER, if you’re conducting an anonymous survey, do NOT pre-populate forms.

4. Send reminders. These reminders can be emails that go out over the duration of the data collection period, or even on-site and social media reminders. And if you really want to boost response rates, send personal emails from personal accounts, not email blasts.

5. Consider appropriate incentives. This is tricky. In order to get the best data, you want people to respond without an ulterior motive. But people really are busy and an incentive — such as a discount off of an event or a free product –can go a long way.

The best way to handle this is to make sure the incentive is small enough to not be the major influencer in people’s desire to respond. Alternatively, offering a free big-ticket item to “one lucky winner” can incite participation but also temper expectations. My favorite incentive, however, is a small-ish guaranteed reward for participating, such as a screen cleaner for smartphones or other small items that can be mailed with little postage. It makes people feel appreciated but is really not enough to influence answers.

Whatever incentive you choose, make sure it’s in line with your brand and geared towards your customers.

Also, where appropriate, offer to share the results with respondents. This is best done by sharing the executive summary of any report, not the complete report, and highlighting that respondents will “Get 10 free charts” in gratitude for their time. Make sure to collect contact information so that you can deliver on your promise. If you’re conducting an anonymous survey, ask for contact information after respondents have submitted their responses, and remind them again on the page asking for their contact information that their responses will not be tied to any identifying information. Charts can also work for customer satisfaction surveys, if you make them public via a blog post and invite more feedback.  

6. Limit survey frequency. Surveys are a serious time investment for your subscribers, and therefore, you shouldn’t demand that much of your audience’s time too frequently. It’s difficult to make hard and fast rules, because perceived annoyance levels are often a function of survey length and audience size. In other words, if you have a small audience, you can send one long survey a year or two to three that take under a minute. If you have a large audience, you can create different surveys for different segments of your audience and then target those prospects only.

You can also embed survey questions on your site and get information that way over a longer period of time. While this is usually not beneficial to most marketers since there’s no way to track audience demographics and group previous answers, subscription sites are uniquely poised to do so because users are registered and need to log in. Survey responses and polls then do double duty as both an engagement tactic that keeps subscribers on site longer, as well as lend you a lot of customer and market data.