A/B Testing

A/B testing also known as bucket testing or split-run testing, is a user experience research methodology. As the name suggests (A and B), two version of a single variable is compared. It is mostly done on digital space. Assuming the current feature A and the new feature to be B. Thereafter, a series of experiments are run to gauge users experience. Let’s take a real scenario. Google ran it’s first A/B testing in the year 2000, though it failed due to technical glitch. The test was done to determine the optimum number of results to display on its search engine page would be. By default google shows 10 results, for more results we need to move to next page. How about increasing or decreasing the results will it impact user engagement? To figure out the answer Google will do some experiment with some existing users and new users to observe the impact of the change in user’s behaviour. That’s how most of the tech companies bring new features/products.

Let’s see another feature of the Google search display. How about changing the link color from blue to red or yellow or your favourite color. Will changing the color impact user engagement? The above questions may sound silly to us. However, even we don’t know the answers. How those features will impact our engagment.

Google and Microsoft runs over 10K A/B testings annually. Just to interpret human behaviour. They gather tons of data to analyse and discover the unknown delicious feature. That’s why data is the only oil in digital world.

A simple A/B testing of changing the ad location on a bing search page result, resulted in an increase in the revenue by the tune of 12%. That too within a few hours. That’s the power of experimenting. Companies keep on running on new experiments everyday just to figure out how to get more attention span from the users that’s where the revenue of tech companies lie.

Ironically at times users may not like that an experiment is being conducted on them to know more about them. That too without letting them know the results 😀

Until then…