By Anya Samek
This post presents results from a large-scale experiment assessing the impact on future donations to charity of thank you calls to past donors.
Thank-you calls are a big deal in the non-profit industry. A google search for fundraising thank-you call yields 47 million results. The top search result is a blog post titled “6 Keys to Rock Thank You Calls and Retain More Donors.” In my own survey of fundraising professionals, 80% said their non-profit conducts thank-you calls at least some of the time, and nearly half said their non-profit conducts thank-you calls often or always. Yet the little that we know about the benefits of thank-you calls comes from surveys of donors or small scale studies. Surveying donors about what they would do can be problematic because when it comes to donation behavior, what people say they will do is not always aligned with what they actually do.
In my study with Chuck Longfield (Blackbaud), we set out to evaluate the impact of thank-you calls at a larger scale, using a randomized design. We conducted experiments with nearly 70 public television stations and a national non-profit over a period of 6 years, involving nearly 600,000 new donors and 500,000 calls made. We randomly assigned new donors to either receive a call or not, and then observed donation behavior one year, two years, and three years later.
Our thank-you calls followed the standard protocols in the industry for large national non-profits. They were made by paid workers and were made 3-7 months following the initial gift. The call script featured three main components. 1) it was personalized, 2) it explained the impact of the gift and 3) it did not involve another request for a gift. The calls reached about 75% of new donors.
Economic theory predicts that thank-you calls could be effective for a variety of reasons. First, personalizing the call primes the recipient’s identity as a donor or member, which can increase giving behavior. Second, explaining the impact of the gift can encourage altruistic donors to give again. Third, because callers did not ask for another gift, the call can be thought of as a ‘gift’ of sorts from the charity, which may encourage donors to reciprocate by giving again in the future.
After conducting our experiments, we carried out two large surveys – one with about 200 fundraising professionals and another with about 5,000 Americans representative of the general population (using the Understanding America Study at the University of Southern California) – to understand whether beliefs about the impact of calls are aligned with the impact of calls observed in our experiments. In these surveys, we told respondents about our experiment and about the donor retention rate in the no-call group, and asked respondents to predict the retention rate in the call group. The surveys involved high stakes, since some randomly selected respondents actually received up to $100 depending on the accuracy of their predictions.
Our results, described in detail here, surprised us. First, we found that our thank-you calls had no impact on donor behavior. Figure 1 provides just one statistic, this is the proportion of donors who donate again in the year following the intervention. As shown by the blue and red bars in the figure, the no-call (control) and call (treatment) group show a retention rate of around 30%. This is surprising given the importance that the fundraising industry places on these calls.
Second, we found that fundraising professionals significantly over-predicted the effect of the thank-you calls. As shown by the gray bars in the figure, the predicted effect of the thank-you call was to increase the probability of donating again to about 50%. Third, fundraising professionals were no better than the public at predicting the effect of the calls, and experience in the fundraising industry was not associated with making better predictions (compare the two gray bars).
Our take-away is if the goal is donor retention, then this method of thank-you calls is ineffective at increasing retention. Thank-you calls could cost around $1 per call, but generate no additional revenue. Charities that wish to continue making thank-you calls may add value if the responses to these calls are used for donor segmentation, since we did find some evidence that people who reacted positively to the calls were more likely to donate in the future.
There is a great deal yet to be learned. Smaller charities often use board members or volunteers to make calls, and the impact of calls made by these groups is an open question. Further, our calls were made months following the initial donation, but calls made closer to the date of the donation could also be more effective.