Comments on: Introduction to A/B and MVT: Optimization Testing 101 http://www.getelastic.com/testing-part-1/ #1 Subscribed Ecommerce Blog Thu, 09 Feb 2012 02:57:16 +0000 hourly 1 http://wordpress.org/?v=3.2.1 By: Hammer Bamharehttp://www.getelastic.com/testing-part-1/comment-page-1/#comment-64485 Hammer Bamhare Sun, 09 Oct 2011 09:25:40 +0000 http://www.getelastic.com/?p=9060#comment-64485 I have just applied for a job asking about A/B and MVT testing. This is a brilliant article. I have just applied for a job asking about A/B and MVT testing. This is a brilliant article.

]]> By: Amithttp://www.getelastic.com/testing-part-1/comment-page-1/#comment-56604 Amit Mon, 25 Apr 2011 11:34:59 +0000 http://www.getelastic.com/?p=9060#comment-56604 Hi Linda, How would you define testing a completely new interface, like twitter did with a headline on top saying "Welcome to #NewTwitter! Read up on what’s new. You can still access old Twitter for a limited time." This way i guess the user has a choice rather than a compulsion to be on one or the other interface. Would this qualify as a Radical Redesign? Thanks. Amit Hi Linda,
How would you define testing a completely new interface, like twitter did with a headline on top saying “Welcome to #NewTwitter! Read up on what’s new. You can still access old Twitter for a limited time.” This way i guess the user has a choice rather than a compulsion to be on one or the other interface.

Would this qualify as a Radical Redesign?

Thanks.
Amit

]]> By: Linda Bustoshttp://www.getelastic.com/testing-part-1/comment-page-1/#comment-25615 Linda Bustos Thu, 29 Jul 2010 00:26:16 +0000 http://www.getelastic.com/?p=9060#comment-25615 Hi Dennis, we're fine with quotation of a paragraph with a link back to our post. In this case, since it's just the introduction, sure go ahead up to Enter A/B and MVT. Hi Dennis, we’re fine with quotation of a paragraph with a link back to our post. In this case, since it’s just the introduction, sure go ahead up to Enter A/B and MVT.

]]> By: Linda Bustoshttp://www.getelastic.com/testing-part-1/comment-page-1/#comment-25508 Linda Bustos Tue, 27 Jul 2010 22:37:49 +0000 http://www.getelastic.com/?p=9060#comment-25508 Cool, thanks for sharing that Matt Cool, thanks for sharing that Matt

]]> By: Amadesahttp://www.getelastic.com/testing-part-1/comment-page-1/#comment-25401 Amadesa Mon, 26 Jul 2010 19:04:11 +0000 http://www.getelastic.com/?p=9060#comment-25401 Great into for A/B and MVT. Regarding fractional factorial tests not being reliable - what would be the point of running an unreliable test? In order to believe the results once the test concludes, reliability is a requirement up front. Regarding your comment "fractional factorial testing methods were developed for the manufacturing industry, where prototypes were expensive to develop. That’s not the case with websites." Exactly! Traffic is [relatively] cheap and making website changes is nothing like changing a physical part in an automobile. Ultimately, the best scenario for MVT is to run the test and automatically remove poor performing variations as they become statistical significant. Enter "automated MVT". See here for more: http://amadesa.com/blog/lala Thanks, Matt Great into for A/B and MVT. Regarding fractional factorial tests not being reliable – what would be the point of running an unreliable test? In order to believe the results once the test concludes, reliability is a requirement up front.

Regarding your comment “fractional factorial testing methods were developed for the manufacturing industry, where prototypes were expensive to develop. That’s not the case with websites.” Exactly! Traffic is [relatively] cheap and making website changes is nothing like changing a physical part in an automobile.

Ultimately, the best scenario for MVT is to run the test and automatically remove poor performing variations as they become statistical significant. Enter “automated MVT”. See here for more: http://amadesa.com/blog/lala

Thanks,
Matt

]]> By: Alvin Tanhttp://www.getelastic.com/testing-part-1/comment-page-1/#comment-25212 Alvin Tan Fri, 23 Jul 2010 06:08:55 +0000 http://www.getelastic.com/?p=9060#comment-25212 Great information as usual. Also, despite the obvious expository, as opposed to argumentative, nature of this article, what would make this article more complete is to link to some of your older articles on the pitfalls of split testing. Anyone who wants to adopt A/B testing should be familiar with A/A testing and the concept of statistical significance before treating the results as useful and actionable. Great information as usual.

Also, despite the obvious expository, as opposed to argumentative, nature of this article, what would make this article more complete is to link to some of your older articles on the pitfalls of split testing. Anyone who wants to adopt A/B testing should be familiar with A/A testing and the concept of statistical significance before treating the results as useful and actionable.

]]> By: Dennishttp://www.getelastic.com/testing-part-1/comment-page-1/#comment-25199 Dennis Thu, 22 Jul 2010 22:50:54 +0000 http://www.getelastic.com/?p=9060#comment-25199 Hi Linda, nice article can I republish the first part (including Enter A/B and MVT) on our blog with a link to this article? Hi Linda, nice article can I republish the first part (including Enter A/B and MVT) on our blog with a link to this article?

]]>