A/B Testing setup guide

Let's set up A/B Testing with Rebuy so you can run experiments, optimize your personalization strategy, and let the data tell you what converts best. A/B Testing makes it easy to pit different versions of your widgets against each other to see which one drives higher revenue or higher conversion rates. Once the test wraps up, the winner automatically takes over for all future traffic. No guesswork, just results. Now here's how to set it up. Before creating an experiment, you'll need at least two different offers to test against each other. If you're testing whether showing a widget outperforms showing nothing at all, then you just need one. We've linked setting up merchandising widgets on this page if you need some help setting those up. Alright, let's dive into A/B Testing. Head to your Rebuy Admin and click Flows and A/B Testing in the left-hand menu, then click A/B Testing. Now if we scroll down here, we can see this is where we keep track of any live experiments we're running, scheduled ones, draft ones, and completed experiments. So let's start a brand new one by clicking Create New Experiment at the top right. Give your experiment a name that's easy to remember, and then select what kind of experiment you'd like to run. We can do Widget, General, Checkout Offer, Smart Cart, and Global Smart Flow. Let's select the Widget example for this one. Now we're in the experiment editor, and here's where we're gonna set up our control and variations. So select your control, which is your baseline. So for this test, I'm gonna see what performs better between two dynamic bundles for the product page. So for my control, I'm gonna select this dynamic bundle right here. And then we can continue to customize this experiment and we can add up to nine variations to test against each other. I'm just gonna choose another variation of this dynamic bundle and see what performs better between the two of them. And we can set the traffic percentage for each. So you might show the control to 50% of visitors and your variation to the other 50%. But if we had multiple variants here and we wanted to weight these, maybe some see more visitors than the others, we can split up this percentage however we would like. It just has to equal 100%. Next, scroll down to Experiment Goal, and this determines how Rebuy picks a winner. Choose Revenue if you want the widget that generates the most dollars, or choose Conversion Rate if you care more about which widget leads to the highest percentage of completed orders. By the way, once you start the experiment, you can't change this end goal, so choose wisely. If you want your experiment to start and stop automatically, you can set specific dates and times in the Test Duration section. Otherwise, you can start and stop it manually whenever you'd like. One thing to keep in mind, if you're using Shopify scripts or custom CSS tied to specific widget IDs, we don't recommend scheduling an automatic end date. That's because when an experiment ends, Rebuy duplicates the winning widget with a new ID, which means those scripts or styles won't carry over. So if that's the case for you, just end the experiment manually so you can handle that transition. Alright, now once everything looks good, click Save Experiment. Our experiment is now in draft mode. Before you can start it, you need to install the placeholder ID on your store. To put it simply, we don't wanna use the widget IDs that were originally created with our control and variant. What we wanna use now is this placeholder ID shown here on the right side of the experiment. That's because this number powers the A/B test to serve the right experiment version to each visitor. So if you're using app blocks, very simple. We're gonna put this placeholder ID directly into the app block where you normally put the widget ID. So let's show you an example of that. With this test we've set up, remember we're seeing which dynamic bundle is gonna perform best on our product page. So let's jump over to Shopify Admin, and then we're gonna click Online Store in the left-hand menu, and then we wanna edit the theme. Okay, now we're in the theme editor, and at the top of our screen, let's click Homepage. And when this dropdown menu comes up, we just wanna navigate to our default product page, and that's where we have that dynamic bundle located. So here's that dynamic bundle, a classic frequently bought together offer. So I'm gonna click this block on the left-hand side, and now see in the top right where it says Widget ID with a series of numbers. Let's replace those with the ones from our experiment. So we'll jump back to the A/B Testing dashboard, grab this number, then paste it in. Perfect. Now all we have to do is set this A/B test live. If you have to edit a theme file for your A/B test, we've left documentation on this page on how to do just that, as well as how to do an A/B test for cross-sell widgets in the Smart Cart. Now since we have that placeholder ID installed on that product page, all we have to do is set this live. So you can start the experiment right here from the editor by clicking Start Experiment in the top right corner. Or we can head back to the A/B Testing dashboard, scroll down, and find our experiment in the Drafts tab and then click the three-dot menu and hit Start Experiment. So if we had scheduled an end time, this experiment would wrap up automatically. Otherwise you can end it any time from the Active tab using again that three-dot menu. Once the experiment concludes, the winning widget will automatically take over and the placeholder transforms to match the winning configuration. No extra work needed. One note, if the winning variation was Do Not Show Anything, then no widget will display. And in that case, you could uninstall the placeholder from your theme since it's no longer needed. Now you can monitor your experiment at any time with the A/B Testing dashboard. Active tests show real-time performance metrics and completed tests live in the Completed tab with full results. Here's an example on our case study page of a brand that tested what converts better, showing reviews in their checkout versus not showing any reviews at all. And as you can see, reviews won. That's a great little pro tip for anyone who isn't leveraging our Checkout Extensions product to add reviews with content blocks. And that's it. You are now set up to run data-driven experiments to optimize your store for maximum revenue. Thanks for watching this video. If you ever get stuck along the way or have any questions, you can reach out to our support team by clicking the Rebuy logo in the bottom right corner of your admin. That's all for this video. We'll see you in the next one.