Let's send a Phrasee test in Leanplum!

At a high level, we're going to use two campaigns and some A/B testing technology that's already baked into Leanplum to execute a Phrasee test.

Begin by creating a Multi-Message Campaign. Give it a unique name and append something to the end of it like Test because we'll be making a copy of it shortly.

Select your Audience and Delivery Method as your normally would.

Then, navigate to the Actions tab and click Add Action.

Choose Email in the left column and then select your template or HTML. Then, click Add Action.

Great! Now, click on the meatballs menu in the upper-right corner. Click Duplicate.

Enter your unique name again but this time add Final to the end of the name.

Click Duplicate. Then, click Close when the success message pops up.

Awesome!

You should still be in your original Test campaign at this point. If you want to double-check, the name of your campaign should appear in the upper-left corner.

Go ahead and click the Target Audience tab located just beneath the campaign name. We're going to select your test population.

Upon import, Leanplum evenly distributes your subscribers amongst user buckets numbered 0 through 999. So roughly, 10 buckets equals 1% of your subscribers.

We're going use those buckets to select a percentage of your subscribers for the Phrasee test.

Under Advanced, click Add Additional Rules.

Then, click Add a rule to start building audience.

Click (you guessed it!) User bucket.

Now, you'll see some boxes appear where you can enter the buckets you want this campaign to target.

To ensure you're choosing enough people to get a statistically significant test, you'll want to use the percentage Phrasee recommended when you made your campaign.

Where can you find this number, you ask? What a great question!

Head back to Phrasee and find the campaign you made for this test. In the upper-right corner, you'll see an arrow button.

Click on that to open a panel.

This panel contains lots of important information about your Phrasee campaign, including the percentage we're after. You'll find it about halfway down the panel under the Details heading.

For our example, our audience is 1,000,000 subscribers. Phrasee has requested 10 test cells of 19,693 people, a total of 19.69% of our audience. Your numbers will very likely differ from this example, so make sure you follow the rest of the directions using your own numbers.

To make it easy, we'll round up to the nearest percentage: 20%.

So, if 10 buckets equals roughly 1% of our Leanplum audience, we would want to select 200 buckets to get 20%.

Leanplum numbers its buckets starting with 0. So for our User bucket is between criteria, we're going to enter 0 and 199. That will select 200 buckets.

Again, your actual numbers will differ from our example.

Once you've entered your numbers, click Apply Additional Rules in the upper-right corner.

Great! With your test audience selected, let's get our test cells set up.

Click on the Actions tab.

Just below the Actions tab, you'll see a button labeled + A/B Test. Click that button now.

You'll notice this has now added another column to your Actions canvas.

We're going to set up nine variants in addition to the control you'll see on the left. Go ahead and click the plus button (+) eight more times until you see Variants B through J.

Now, click on the gear icon to the left of your variants.

In the window that appears, click Divide equally so that each variant gets roughly the same amount of subscribers.

Click Save.

Now, we'll add your Phrasee language to each of your test cells. Back in Phrasee, you can click on the copy icon next to each variant to copy it to your clipboard.

Then, simply return to Leanplum and paste it into the Subject field for each variant. Make sure you've selected the proper email action and variant in the Action canvas.

For example, for the Control, you'll need to click on the email itself in the Control column.

But for the variants, you'll need to click on the email itself in the Variants column and the variant name. When you do this, you'll see the email's outline will change to match the color of the variant as shown below.

Once you've pasted in all of your variants, click the gear icon again.

We're now going to add a bit of the language to each variant name so you can easily tell which is which is the reporting view later.

When you're finished your tab will look something like the below.

You can now finish checking your campaign over and send it out as you normally do.

Note: For this testing methodology, it's important all subscribers get the message at the same time so each variant has an equal chance of receiving an open. So ensure you are using a send method that deploys to everyone at the same time.

After your email has deployed and you wait the predetermined number of hours for your test results to mature, return to your campaign and follow these steps to determine your winner and export your data.