To blurb or not to blurb?

Below is an experiment run by Ben Raue at GetUp to see whether text blurbs on donation forms improved action-rates. If you have done a similar experiment please let us know.

Experiment design

Our donation form includes a section near the top where the campaigner can insert some explanatory text. This can be text required for legal or regulatory reasons, or an explanation of the campaign to complement the campaign page that sits in the main body of the page (with the donation form on the right-hand sidebar).

For this experiment, some visitors to the website were randomly allocated to a donation page which did not include any text at the top of the donation form. The two versions of the form can be seen below:

Top-line experiment results

Combining all visitors to the website who participated in the experiment, the experiment produced the following results.

This result is close to being statistically significant. Unfortunately, the gap has closed slightly since preliminary analysis.

Detailed experiment results

We have also used Google Analytics to track the experiment results, broken down by different donation pages.

The problem with this experiment is that each donation page has different words in the blurb, and the content on the main body of the page varies from page to page. It appears that the removal of the blurb is more useful on pages with a relatively short body text.

There are five pages on our website which raised significant amounts of money during this experiment. The results vary significantly between pages.

“Thanks for completing our Vision Survey!”

The body text on this page is very short. The inclusion of a blurb can push parts of the donation form below the bottom of the browser on many screen. The removal of the blurb removes the need to scroll down, and we see a big increase in the proportion of visitors donating.

“Help us fight Adani”

The body text on this page takes up about as much vertical space as the donation form without the blurb, so the inclusion of a blurb may require otherwise unnecessary scrolling. We saw a large increase in the proportion of donations.

“Make sure our Senate see this ad”

On this page, the inclusion of the ad image means that the body of the page covers a larger area. If you scroll down to read the ad, the version of the donation form including the blurb is perfectly lined up. If anything, the donation form without a blurb is too high. This resulted in slightly more donations from the control.

“Chip in to get this ad in the paper!”

The page text is substantially longer than the donation form, so a person who reads to the end can no longer see the donation form. The proportion of donations is about the same between the two versions.

“Let’s get loud: will you help slam the brakes on the TPP”

This page also contains much more text, and we see little difference between the control and variation.

Conclusion

While the overall results suggest a moderately positive impact for removing unnecessary text from the donation form, the more detailed results suggest a more complicated picture.

In cases where the main body of the page contains a lot of text or images, there is no benefit from removing text on the donation form, and it’s possible that a donation form that extends to the bottom of the page could work better.

Recommendations

1. Create an alert for campaigners when making a donation form if they include a blurb over a set character limit, to encourage them to think carefully about making this form longer.
2. Investigate making the donation form “sticky” so it moves with the rest of the donation page if the user scrolls.