Talk:A/B testing/Archives/2013

This page is an archive of past discussions. Do not edit the contents of this page. If you wish to start a new discussion or revive an old one, please do so on the current talk page.

How is A/B Testing different from an experimental

Indeed, A/B testing is just one kind of group experimental design. However, it is rather more specific in its description, and is very widespread in the world of commerce. For at least these two reasons, it deserves its own article, not just a Redirect entry. Probably, there should be wikilinks to this article in several other, more general articles. By the way, this article needs more references. David Spector (talk) 13:49, 13 March 2013 (UTC)

I have reverted the deletion of many testing products because the deletion was not justified in this Talk page. The editor claimed that only notable tools were spared, but did not give convincing evidence. WP is not in the business of recommending some products over others. David Spector (talk) 16:34, 17 June 2013 (UTC)

Hi, my approach is to provide internal wikilinks to notable products, where notability is manifest by having an encyclopedia article for that product. Otherwise we will have a messy sprawling linkfarm directory, which is not what Wikipedia is supposed to be, per

WP:LINKFARM. So far, the products with WP articles are retained. Anyone can add more products once they write notable articles about the products. Cheers. Logical Cowboy (talk

) 17:11, 17 June 2013 (UTC)

I see your point, and it does makes sense now that you've explained it. I don't think it is an ideal solution, but WP policies, sensible though they seem individually, do not automatically give us an encyclopedia that is accurate, reliable, and comprehensive, all at the same time. David Spector (talk) 00:09, 18 June 2013 (UTC)

"Companies well-known for using A/B testing" looked like advertising. Please explain what's exceptional about them using a/b testing, if you want to add that again. As comparsion there is no section "Companies well-known for using brain teasers.". — Preceding unsigned comment added by 89.204.138.10 (talk) 14:18, 6 August 2013 (UTC)

I've read this article three times now and I still have no idea what A/B testing is. I wonder if I'm the only one with this problem. -- Christopher C. Parker ^{t c} 13:57, 15 July 2010 (UTC)

I second that!(I still have no idea what A/B testing is!) I followed a link to the article and I am sincerely clueless as to what that was supposed to have explained. Maybe an example or two could help. 208.96.185.47 (talk) 10:31, 7 August 2010 (UTC)

Triple that. I was referred to here for an explanation, but I only got some vague clues. If it would be of any help to future editors, here are some of the particular things I failed to understand:

• baseline control sample - is this something the issuer of the test knows something about, like these particular apples sell very well, or what?
• single-variable test sample - is this the same as that baseline, but with a single feature changed?
• response rate - what is a response? People buying the product, saying positive things about it, or simply noting its existence?

• Significant improvements can be seen through testing elements like copy text, layouts, images and colors

  - do two identical leaflets printed in a different colour constitute an A/B test?

• distribute multiple samples of a test, including the control, to see which single variable is most effective

  - so, a brochure having several variations, each in a different colours but otherwise identical to the others, will be distributed to a large group so that each individual gets exactly one copy, and the observation would be which colour "does it the best"? Or an individual will receive several diffeerently coloured copies and asked to express an opinion?

Sorry if any of the above is obvious to all except me, but this is what I roughly understood. 78.90.134.151 (talk) 20:14, 10 August 2010 (UTC)

Too bad the article doesn't answer these, but I'll answer here for you:

-baseline control sample: usually the original version. It's a baseline because it's what you start with before you do any test, and you already know the resulting behaviour of it. It's a control because it's not changed, and you are testing a change.

-single-variable test sample: yes, a version of the original, ideally with just one single thing changed (you'll see people changing one TYPE of thing rather than one particular thing though, such as changing all the content on a web page (but not the graphic design or how the back end works)). This is offered to the B group.

-response rate: this is based on what the goal of the thing you're testing is. IF your goal is buying the product, then that's what you use. But on web pages, for example, it's often nothing more than "does the user click on the next page?". If it's a brochure, is your goal to get people to call the telephone number on there? You the tester decide what constitutes a response.

"Do identical leaflets printed in a different colour constitute an A/B test?" Yes, IF you are testing a specific response to that colour change, and are giving out both the original (baseline control sample) in the same amount as the changed version (single-variable test sample).

"Or an individual will receive several diffeerently coloured copies and asked to express an opinion?" No. Each participant receives only one copy and does not see any other versions. They are not asked what they think, but simply their behaviour is observed.

A/B testing is generally a either-or behavioural test. It's just one test of many different sorts you would do, so A/B testing is usually part of a suite of tests. 188.204.147.121 (talk) 12:36, 28 March 2011 (UTC)

A crystal-clear explanation

"I've read this article three times now and I still have no idea what A/B testing is."

Most commercial websites have some action on the part of the visitor as their goal, which is also a business goal for the company. For example, they may aim to have the visitor sign up for a free Web-based service that they offer, or they may aim to sell any of a long list of products.

These websites use fundamental Web-design elements such as page layout, graphics, text, and interaction in an attempt to move the visitor (or user) to take action that moves them toward the website's goal.

The designers of the websites are only human, so they don't know exactly which design elements will work best, which will cause the most visitor actions to be taken that meet the business goal.

One way to design such a goal-directed website is to create two separate designs (called, perhaps, "A" and "B"), differing in only one specific design area, and present one or the other design to each visitor, randomly.

For example, one design may offer a limited-time discount to those visitors who sign up for the service or make an order. The other design does not offer this incentive.

During the visitor's action, the website internally sends itself a random number or asks the user to include some brief information to indicate which of the two random designs was presented to this particular visitor.

When the website interacts with the user, then, it can measure whether the user did or did not take a goal action, and it also knows which of the designs was presented.

Over time, statistics can then be automatically generated showing the percentage of effectiveness at achieving the goal of each of the two designs. If the statistics show that one design is significantly better than another, that design can be made standard and the process repeated with a new design change.

Gradually over time, then, the site's design can be optimized to work best to achieve the site's business goals. Larger lessons can also be learned about what kinds of design elements work best.

Although this description has been presented in terms of websites, the same principles are used to optimize response rates from mailing campaigns, TV advertisements, and other kinds of commercial interactions with consumers. David Spector (talk) 13:41, 13 March 2013 (UTC)

Still unclear

But HOW does the test work? You randomly assign n people to group A and n people to group B. You then observe which group has a better success rate (website visits, product sales, or whatever your goal is). When is the difference significant? How to test for it? Isn't this A/B test really nothing else than the

I made an image on AB testing for our website http://www.reedge.com/ab-test.html and I can remove our logo and donate it for this article if it helps. I would need someone to add it for me sice I am not a Wikipedia expert. It might help explain the concept of AB testing a bit. Dennis van der Heijden

Please go ahead and create the image. One of our editors would gladly upload the image on your behalf. ♠♠ BanëJ ♠♠ ^(Talk) 04:07, 7 March 2012 (UTC)

I removed the logo myself and have the image ready. If you can donate it, it would be great as I really think the image is perfect. It would provide a much clearer explanation of A/B testing (especially within the context of web sites). Elvanor (talk) 21:04, 12 April 2012 (UTC)

I uploaded the image to the new site of Convert.com (was Reedge.com) and will donate it to the A/B testing Wikipedia page, link is http://www.convert.com/wp-content/uploads/2013/09/ABTesting-838x1024.png Maybe someone can upload it to the page... I'll try, but not sure how. Dennis van der Heijden 4 September 2013. —Preceding undated comment added 04:37, 5 September 2013 (UTC)