Asking About Improvement: Survey Tips

Above all else, nonprofits want their evaluation efforts to “strengthen their future work.” I know from my experience helping nonprofits develop and use an evaluative lens that most value their stakeholders’ feedback as a crucial part of achieving this goal.

Yet they tend to make one critical mistake. They ask on a survey, “What could we do to improve?” Stakeholders appreciate being asked for their opinion. But generic, open-ended phrasing is difficult for them to answer. Nonprofits also struggle to make good use of their answers.

Below are three problems that arise when nonprofits pose this question, as well as three alternative ways to ask for – and use – stakeholders’ feedback about improvement:

Problem 1: It unfairly fishes for ideas

Programs and organizations often ask, “How can we improve?” when they’re short on ideas themselves. Thinking about how to improve your program, obviously, is hard work. But asking participants for their suggestions about improvement unfairly pushes the responsibility of improving your program onto them. It asks them to imagine something that doesn’t exist: your program improved. And because open-ended questions – by definition – are hard, many people won’t give this one their full attention. The quality of their responses will be low.

Solution: Test specific options

Use your stakeholder survey to detail realistic programming options. Ask your stakeholders – in a multiple choice format – which they would prefer. This communicates to them that you value their opinions while putting the responsibility (and the credit) for developing a realistic vision on you, the nonprofit. You’ll find that when you do this hard work ahead of time, you’ll ultimately gather more useful information.

Example:

1. We have the ability to add 30 minutes to this program.
Would you prefer that we add 30 minutes or keep it the same?
     o Add 30 minutes
     o Keep the length the same
     o Either is fine with me

2. If yes, would you prefer that we spend that time on additional training or additional time for group work?
     o Trainings
     o Group work

Problem 2: It focuses on problems instead of improvements

A problem is not the same thing as an improvement. While the question, “What could we do to improve?” is ostensibly about program improvement, I often see organizations treating it as a way to determine stakeholder satisfaction. If few unhappy people respond, the thinking goes, the program is fine and does not need improvement. This is a false assumption. It’s wrong to think your program can’t be improved if no one identifies a problem.

When stakeholders are unhappy, on the other hand, they tend to write detailed accounts of their unique problem. Sometimes these unique responses help identify a potential improvement but often they are wholly dismissed because “Well, that’s just one person” or “Too bad, that problem is out of our hands.”

Solution: Look for themes

If you’ve already asked, “How are we doing,” look for themes in unique responses. You may find a common theme. For example, one person complaining about the room and another complaining about parking fit into a theme about the location of the program. Even if you find contradicting responses in a theme (e.g. some people said the room was “great!” and some said, “Terrible”), you can turn this into a more specific, actionable question on the next survey.

Problem 3: It creates unrealistic expectations

Asking “How can we improve?” tells your stakeholders that you are willing to use their feedback to make program changes. But what if someone suggests ideas that are beyond your means or control? You’ve just given them a reason to disengage.

I also see organizations ask the improvement question even when they aren’t in the process of improving their program. If your plan is to do nothing with the responses unless “something interesting comes out of it,” just skip the question. It’s false advertising.

Solution: Ask in a different way

Testing options is the best way to get useful feedback from stakeholders and set realistic expectations. But nonprofits can’t think of everything. Asking an open-ended question can bring to the surface hard-to-see details, perspectives, or problems. Instead of asking “what could we do to improve,” however, try ending your survey with, “Is there anything else you would like to tell us about this program?”

This wording elicits open-ended feedback without setting up an explicit expectation that you will use it. This is also a good way to ask for ongoing feedback even if you aren’t in the process of improving your program. If responses start turning up common themes, turn them into survey questions for the future! Your stakeholders will see you being proactive rather than unresponsive.

Conclusion

Improving a program is difficult work. First, decide if you are ready and willing to make program changes based on stakeholder feedback. If not, that’s okay. Just don’t ask about what you could improve. Instead, give respondents a chance to rate their satisfaction or enjoyment. Then follow up with, “Is there anything else you would like to tell us about this program?”

If you do decide you’re ready to change your program, do the hard work of coming up with realistic options. Use your survey questions to test those options with your stakeholders. The responses will be more useful to you and your stakeholders will thank you.

Image: “Hot Air Balloon Man” by richardstowey is licensed under CC BY 2.0 and has been cropped. 

Creative Commons License This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.