Really? You Want to Ask That? - Creating surveys that capture the data you need

Really? You Want to Ask That? Creating surveys that capture the data you need
October 2015 by Donna Bellew

Recently I had dinner at restaurant not far from my house. Within minutes of finishing my meal and paying the bill, my phone beeped: I had received a request from the restaurant soliciting feedback. I was pleased to be asked. Dinner had been delicious and I hoped my feedback could help this good business be even better. Unfortunately, things went south from there. The survey was long, the questions were confusing and many questions referred to food and people I hadn’t encountered on my visit. Completing the survey flipped my positive impression of the restaurant into a negative one and as I progressed through the long survey I suspect my feedback reflected this.

With new technology it has never been easier to gather and evaluate information from a myriad of sources. Collecting information can bring our stakeholders closer and give us valuable information to make important decisions. But at the same time, poorly designed surveys can wear our clients’ patience and frustrate the very people we are hoping to serve better.

But putting together a great survey is tough. It takes time and thought. That’s the bad news. The good news is your supporters, clients, and stakeholders want to help your organization be stronger and by following some simple tips you can put together a great survey that will help them to help you.

Getting Started

Know What You Want to Know

What you want to know is the most important part of survey design. As you draft each question ask yourself:

  • What will we do with this information?

  • Will asking the question this way give us the information we need?

  • Will this information help us make our important decisions?

Your survey should focus on answering those questions. Don’t fall into the “while we are at it, let’s also ask about . . . ” trap and add too many questions. Be strong and cut all questions that are not on point. Having fewer questions with more focus will improve the depth and quality of your responses

Confidentiality

Be clear about confidentiality. Tell respondents how and with whom results will be reported. This is particularly important with regards to comments. For instance, will comments be shared verbatim or only paraphrased? If commenters give their name will that potentially be published? In small organizations, if you have asked for details such as role or length of time served, it may be possible to identify exactly who said what if the full data set is shared. Think through these issues and be clear from the start about who will have access to the results and who will not.

Under 15 Minutes Please

No one has ever said, “I wish this survey was longer”. Research suggests that response rates will start to go down in as little as 5-10 minutes. After 15 minutes respondents have a high abandonment rate, stop paying attention, or worst case, become angry. You value the people you are surveying, so use their time wisely.

Writing Great Questions

Your results are only as good as your questions. Here are some tips for writing clear questions that will get you results you can use:

Simple and Specific

Problematic: I am confident in my ability to communicate effectively.

Better: I am confident in my ability to present my findings at a conference.

 

One Idea Per Question

Problematic: How satisfied are you with the hours and location of our office.

Better: How satisfied are you with our office hours?

Better: How satisfied are you with our office location?

 

Avoid Leading/Biased

Problematic: Fundraising is hard. Did the great donor workshop make you better prepared for the long arduous upcoming campaign?

Better: Did the donor workshop prepare you for the upcoming campaign?

Strongly Agree Agree Disagree Strongly Disagree

 

Avoid Double Negative

Problematic: Do you agree or disagree that our organization should not ask for gifts more than twice a year?

Strongly Agree Agree Disagree Strongly Disagree

Better: How frequently should our organization ask for gifts?

0-2 times a year 3-5 times a year as frequently as needed

 

Other Best Practices

Start Easy- use the first part of your survey to ask easy questions. Save tough and sensitive questions for later in the survey.

Use Scales- scales like strongly agree to strongly disagree allow you to measure intensity and provides more detailed information. Use them when you can.

Ask for Specifics-if you want details, ask for them. Offer a comment box and ask, “We would love to know more about your thoughts on….”

Define your terms-words like “most” and “numerous”, mean different things to different people. Use specific ranges like “1-2 days a week, 3-5 days a week etc.” when possible

Utilize Contingency Questions- this allows for more detailed responses. For example: “Did you attend our spring fundraiser?” If the respondent answers “Yes” she is then asked “What did you like about it?” If the respondent answers “No” she is then asked “Why not?”

Most Important

Pilot Test your Questions-send a draft of your survey to a close group of testers to make sure your questions make sense and give you the type and quality of information you are seeking. You will be amazed how often something that seems obvious to you is interpreted completely differently by people taking your survey.

Maximize Response Rates

Great questions are half the battle. The second half is ensuring a great response rate.

Remember the people you are sending your survey to want to help you. They are your supporters, clients and stakeholders so they have a vested interest in your success. With a little planning and careful consideration it can be easy and enticing for them to help you by responding.

You want respondents to want complete your survey. To do this, consider the following:

  • Who answers your survey is as important as what you survey. Choose your respondents wisely. Make sure the survey is relevant to them. If you survey about a recent event, send your survey only to those that attended, not your whole list.

  • Let people know why you are asking for their particular input. Tell them why you want and value their opinion.

  • Enlist staff and board members to follow-up with individuals they know and encourage responses. This is a great way for the board to contribute to the success of the survey.

  • Make sure to let respondents know how their results will be used to improve the organization

Ready. Set. Survey!

You are now prepared to create a great survey and use the results to drive change in your organizations. Remember…

  • Know who you want to ask and why

  • Keep it short and simple

  • Test your questions

  • Personalize requests for responses

Add new comment

Filtered HTML

  • Web page addresses and e-mail addresses turn into links automatically.
  • Allowed HTML tags: <a> <em> <strong> <cite> <blockquote> <code> <ul> <ol> <li> <dl> <dt> <dd>

Plain text

  • No HTML tags allowed.
  • Web page addresses and e-mail addresses turn into links automatically.
  • Lines and paragraphs break automatically.
CAPTCHA
This question is for testing whether or not you are a human visitor and to prevent automated spam submissions.