split testing

I wrote a split-testing plugin (also known as “A/B testing”) for my CMS early last year. It wasn’t very good, so I wrote an external split-testing tool instead, which you can you for your own websites if you want.

Split-testing application

It works similarly to Google’s “web optimiser”, but I think it’s easier to use.

As a test, I chose to optimise the front page of KV Sites, to see if I could encourage people to contact me or read other pages on the website.

In particular, I wanted to see if it was better to explain what I do, or to assume that the reader knows and just wants to get on with.

As an example, consider the following two extracts:

Indirect speach – explaining what I do
Web Development
Reduce costs by automating and networking your business.
Proven record, with a number of large projects completed. Example projects include Duffy Transport's management application, the crop prediction software used by Cropworks, ongoing development of the WebME CMS, and technical support for a number of projects by other web development companies.

vs.:

Direct speech – asking what the reader wants
I am looking for Web Development
If you are looking for an online solution which to reduce your office expenses, we have experience writing solutions which can cut hours of work per day off your load. Read more about our web application development here.

It seems obvious that the direct speech version would work better, but when your job is involved, it’s better to be certain than to work based on assumptions.

So, I set up a split test. On the homepage of KV Sites, I sent the HTML for both versions to the browser, and set the different versions to display randomly using CSS.

Each version includes a “more info” link (of sorts), so I added the “conversion” code (which records when people go to the page you want them to) to the pages they were linked to, and started recording.

It took a few weeks for the information to come in, as my site does not have heavy traffic, and I didn’t want to artificially boost it in any way, but here’s how it turned out:

Name Visitors Conversions
front page direct marketing 237 v2: 12
v1: 24

It’s obvious from the above that “v1” (the direct speech one) is twice as effective as “v2” (indirect speech).

I’m very encouraged by these results. They mean that it is possible to both a) improve “conversions” based on text changes, and b) provide numbers proving those results.

Leave a Reply

%d bloggers like this: