Multivariate testing in marketing
In
In internet marketing
In
Multivariate testing is usually employed in order to ascertain which content or creative variation produces the best improvement in the defined goals of a website, whether that be user registrations or successful completion of a checkout process (that is,
Testing can be carried out on a dynamically generated website by setting up the server to display the different variations of content in equal proportions to incoming visitors. Statistics on how each visitor went on to behave after seeing the content under test must then be gathered and presented. Outsourced services can also be used to provide multivariate testing on websites with minor changes to page coding. These services insert their content to predefined areas of a site and monitor user behavior.
In a nutshell, multivariate testing can be seen as allowing website visitors to vote with their clicks for which content they prefer and will stand the most chance of their proceeding to a defined goal. The testing is transparent to the visitor with all commercial solutions capable of ensuring that each visitor is shown the same content on every visit.
Some websites benefit from constant 24/7 continuous optimization as visitor response to creatives and layouts differ by time of day/week or even season.
Multivariate testing is currently an area of high growth in internet marketing as it helps website owners to ensure that they are getting the most from the visitors arriving at their site. Areas such as
There are two principal approaches used to achieve multivariate testing on websites. One being Page Tagging; a process where the website creator inserts JavaScript into the site to inject content variants and monitor visitor response. Page tagging typically tracks what a visitor viewed on the website and for how long that visitor remained on the site together with any click or conversion related actions performed. Page tagging is often done by a technical team rather than the online marketer who designs the test and interprets the results in the light of usability analysis.[5] Later refinements on this method allow for a single common tag to be deployed across all pages, reducing deployment time and removing the need for re-deployment between tests.
The second principal approach used does not require page tagging. By establishing a DNS-proxy or hosting within a website's own
Multivariate testing can also be applied to email body content and mobile web pages.
In addition to testing the efficacy of various creative/content executions on a website, the principles of multivariate testing can and often are used to test various offer combinations. Examples of this are testing various price points, purchase incentives, premiums, trial periods or other similar purchase incentives both individually and in combination with each other. The value of this is that marketers (both traditional and online) can use multivariate testing principles online to quickly ascertain and predict the effectiveness of offers without going through the more traditional multivariate testing methods which take significantly more time and money (focus groups, telephone surveys, etc.).
Design of experiments
This article may contain verify the text.(January 2008) ) |
Statistical testing relies on design of experiments. Several methods in use for multivariate testing include:
- Full factorial the most straightforward method whereby all possible combinations of content variants are served with equal probability.
- optimal design discern.[6]
- Optimal design involves iterations and waves of testings. Optimal design allows marketers the ability not only to test the maximum number of creative permutations in the shortest period of time but also to take into account relationships, interactions, and constraints across content elements on a website.[citation needed] This allows one to find the optimal solution unencumbered by limitations.
- Taguchi methods: with multiple variations of content in multiple locations on a website, a large number of combinations need to be tested, and medium/low traffic websites can take a long time to get a large enough sample to find statistically significant differences in performance if differences really exist. For example, if three different images are to be tested in each of three locations, there are nine combinations to test. Taguchi methods (namely Taguchi orthogonal arrays) can be used in the design of experiments in order to reduce the variations but still give statistically valid results on individual content elements.[7] Taguchi uses fractional factorial designs.
See also
- Web usability
- Multivariate statistics
- A/B testing
- Cultural multivariate testing
- Product optimization
References
- ISBN 3-211-83473-7.
- ^ "What is multivariate testing?". www.campaignmonitor.com.
- ^ "Experimentation & Testing: A Primer". Avinash Kaushik. 2006-05-22.
- ^ "WilsonWeb.com, Conversion/Testing: 10 Factors to Test that Could Increase the Conversion Rate of your Landing Pages, by Sumantra Roy, 06/05/2007".
- ^ "Archived copy". Archived from the original on 2008-08-28. Retrieved 2008-08-20.
{{cite web}}
: CS1 maint: archived copy as title (link) "Web Analytics Demystified", "Web Analytics and Data Collection: The Page Tag", By Judah Phillips - ^ MarketingNPV, 3 Ways to Accelerate Your Learning Process
- ^ Webpronews.com, Scientific Web Site Optimization using AB Split Testing, Multi Variable Testing, and The Taguchi Method, by Matthew Roche, 07/26/2004 Archived 2007-11-04 at the Wayback Machine