Talk:A/B testing/Archives/2019

This page is an archive of past discussions. Do not edit the contents of this page. If you wish to start a new discussion or revive an old one, please do so on the current talk page.

External links modified

Hello fellow Wikipedians,

I have just modified one external link on A/B testing. Please take a moment to review my edit. If you have any questions, or need the bot to ignore the links, or the page altogether, please visit this simple FaQ for additional information. I made the following changes:

Added archive https://web.archive.org/web/20150921174256/https://developer.amazon.com/public/apis/manage/ab-testing/doc/math-behind-ab-testing to https://developer.amazon.com/public/apis/manage/ab-testing/doc/math-behind-ab-testing

When you have finished reviewing my changes, you may follow the instructions on the template below to fix any issues with the URLs.

This message was posted before February 2018.

regular verification using the archive tool instructions below. Editors have permission to delete these "External links modified" talk page sections if they want to de-clutter talk pages, but see the RfC before doing mass systematic removals. This message is updated dynamically through the template {{source check

}} (last update: 18 January 2022).

If you have discovered URLs which were erroneously considered dead by the bot, you can report them with this tool.
If you found an error with any archives or the URLs themselves, you can fix them with this tool.

Cheers.—InternetArchiveBot (Report bug) 19:16, 23 June 2017 (UTC)

The rest of the story

I'm not asking a question for discussion here on the talk page, but rather asking for an article improvement.

Consider this scenario:

Web page "A" is seen by 100 people. 50 continue to purchase the product. But the analysis should not end there, because of the remaining 50, 25 come back the next week and purchase the product. Total sales: 75.

Web page "B" is seen by a different group of 100 people. 60 continue to purchase the product. Hurrah, page "B" is victorious! But the obnoxious aggressive advertising of page "B" is so annoying that the remaining 40 vow never to return. Total sales: 60.

How is this accounted for in A/B testing? 72.208.150.248 (talk) 15:54, 1 March 2017 (UTC)

You can set the sales as your conversion objective, so you'll see 75 conversions in Page A and 60 sales in page B, that's a matter of what you are meaning to optimize and it's very important to get this right, optimizing for the wrong metric can be very hurtful to your business.

--Sicarul (talk) 00:10, 1 July 2017 (UTC)

= Common Test Statistics

Gibbs sampling is not in any sense a test statistic. It's a simulation method, and it's generally not even going to be a sensible approach for simulating the distribution of whichever test statistic you might want to insert here; generally straight simulation will be easier and faster. It literally has no place in this table - its presence here is a category error. Further, if a distribution is actually unknown how are you going to write a Gibbs sampler (or indeed any kind of simulation) for it? Bootstrapping (or a permutation test, or some other form of resampling) could work as an approach if you had a statistic but bootstrapping wouldn't belong in the table either -- whatever test statistic you bootstrapped would. Glenbarnett (talk) 22:38, 7 May 2019 (UTC)