Skip to content
Avetti Support
Avetti Support

Getting Started

  • Getting Started eShop
  • Getting Started Marketplace
  • Getting Started Multi Store Ecommerce
  • Getting Started With Accessibility
  • Getting Started With Website Performance
  • Getting Started With Google Analytics
  • Getting Started With Microsoft Clarity

Admin Pages

  • Create a New Supplier Store
  • Setup the Template/Default stores
  • Log in to a Supplier store
Dashboard
  • Dashboard
Catalog and Order Management
    Catalog
    • Product Dimensions
    • Sync Ecommerce
    • Find Products to Sell
    • Inventory
    • Prices
    • Manage Products
      • Products
      • Create Product
      • Edit Product
      • Edit Product Variations
    Orders
    • Quote Request
    • Chat
    • Return and Refunds
    • Click & Collect Orders
    • Manage Orders
About Me
  • General Information
  • Pickup Locations
  • Payment Information
Marketplace
  • Login Access Codes
  • Manage Standard Entities
  • Marketplace Category Mapping
  • Rebuild Marketplace
  • Category Profit Margin
  • Manage Supplier
  • Approve Supplier Data
  • Auction
Appendix
    Documentation Slides
    • About Me
    • Add Items – Suppliers
    • Create a Merchant Store
    • Customers
    • Manage Orders – Supplier
    • Prices – React Admin

System Admin Pages

  • Avetti Commerce Features
  • Ship Station Integration
B2C -Overview
    Templates
    • Catalog Personalization
    • Reset CDTO Cache
    • Template Skins
    People
    • Allowances
    • Customer Activity
    • Customer Properties
    • Customers
    • Member Types
    Marketing
    • A/B Split Tests
    • Add-ons
    • Basket Additions
    • Bundles
    • Coupons
    • Customer Discounts
    • Customer Segments
    • Customer Segment Prices
    • Customer Segment Ads
    • Gift Cards
    • Gift Certificates
    • Promotions
    • Sales Area Geolocation
    • Upsell/Cross-sell
    • Viral Promotions
    Settings
    • Basket Preferences
    • Image Scaling
    • Item Properties
    • Languages
    • Tax Rates
    • Order Preferences
    • Page Parameters
    • Payment Gateways
    • Retail Location
    • Search Dictionary
    • Shipping Countries
    • Shipping Packages
    • Shipping Rates
    • Sitemaps
    • Store Currencies
    • Shipping Exceptions
    • Store Preferences
    • Store Properties
    Audit Logs
    • Audit Log Views
    Catalog
    • Categories
    • Downloadable Items
    • Inventory
    • Item Reviews
    • Item Size/Weight
    • How to attach products to a category in the MCW
    • Manage Facets
    • Manage FAQs
    • Manage Galleries/PDFs
    • Manage Products
    • Multilingual Content
    • Prices
    • Validate Catalog Data
    • View Store
    • Variations
    Advanced
    • CDTO Editor
    • Change Password
    • Password Security Rules
    • Reports Configuration
    • Shopper Pwd Security
    • Site Admin Preferences
    • Solr Categories
    • Solr Items
    Media
    • Manage Media
    Orders
    • Abandon Cart
    • Customer Messages
    • Email State
    • Manage Orders
    • Manage Refunds
    • Reports
B2B - Overview
    Marketplace
      Advanced
      • Solr Items
      • Solr Categories
      • Site Admin Preferences
      • Shopper Pwd Security
      • Reports Configuration
      • Password Security Rules
      • Change Password
      • CDTO Editor
      Catalog
      • Item Status Settings
      • Item Options
      • Prices
      • Manage Facets
      • Item Size/Weight
      • Item Reviews
      • Inventory
      • Categories
      Marketing
      • Viral Promotions
      • Upsell/Cross-sell
      • Shipping Exceptions
      • Sales Area Geolocation
      • Marketing Segments
      • Gift Certificates
      • Gift Cards
      • Customer Segments
      • Customer Segment Prices
      • Customer Segment Ads
      • Customer Retargeting
      • Customer Discounts
      • Coupons
      • Bundles
      Market Place
      • Notifications
      • Master Catalog Products
      • Marketplace Suppliers
      • Marketplace Payments
      • Marketplace Categories
      • Category Profit Margin
      • Approve Products
      Media
      • Manage Media
      Order
      • Manage Refunds
      • Manage Orders
      People
      • Customers
      • Customer Properties
      Settings
      • Basket Preferences
      • Image Scaling
      • Item Properties
      • Order Preferences
      • Payment Gateways
      • Retail Location
      • Tax Rates
      • Store Preferences
      • Store Currencies
      • Shipping Countries
      Templates
      • Manage Templates
      • Reset CDTO Cache
      • Catalog Personalization
    Suppliers
      Catalog
      • Item Properties
      • Inventory
      • Manage Products
      Market Place
      • Approve Products
      • Marketplace Payments
      • Notifications
      Media
      • Manage Media
      Order
      • Customer Messages
      • Reports
      • Manage Refunds
      • Manage Orders
    MCW
      Advanced
      • Change Password
      Market Place
      • Notifications
      • Marketplace Suppliers
      • Approve Products
      Media
      • Manage Media
      Order
      • Customer Messages
      Settings
      • Store Preferences
      • Store Currencies
      • Shipping Countries
      • Item Properties
      Catalog
      • Categories
      • Inventory
      • Item Options
      • Item Reviews
      • Item Size/Weight
      • Item Status Settings
      • Prices
B2B2C - Overview
  • B2B Integration
Import and Export Files
  • Export Orders
  • Import Price and Inventory
  • Importing/Exporting Shipping Rates Spreadsheet
  • Import/Exporting SpreadSheets
  • eShop Features
Appendix
    Documentation Slides
      Others
      • Creating a Supplier Store
      • Rebuild solr
      • Publish
      People
      • Customer Data Management
      • Customers
      • Customer Properties
      Settings
      • Basket Preferences
      • Item Properties
      • MULTI-VALUED PROPERTIES
      • Retail Location
      • Shipping Countries
      • Sitemaps
      • Store Currency
      • STORE PREFERENCES
      • Store Properties
      • Tax Rates
      Advanced
      • Site Admin Preferences
      Reports
      • Business Reports
      Catalog
      • Advanced edit
      • Categories
      • Categories Spreadsheet
      • FREQUENTLY BOUGHT ITEMS
      • Import from XLS
      • Import-Export Advanced Spreadsheet
      • Inventory
      • Item Reviews
      • Manage Facets Manager
      • Simple Add
      • Prices
      • Validate Catalog Data
      • Variations
      Marketing
      • Bundles
      • Basket Additions
      • Coupons
      • Customer Discounts
      • Customer Segments
      • Gift Cards
      • Gift Certificate
      • Promotions
      • Upsell-Cross sell
      Marketplace
      • Approve Products
      Templates
      • Reset CTDO Cache
      Orders
      • Customer Messages
      • Export Orders
      • Email Settings
      • Manage Orders
      • SMS Settings
    Documentation Videos
    • Payment Gateway
    • Retail Location
    • Validate Catalog Data
    • Site Admin Preferences
    • SMS Settings
    • Advanced Edit
    • Variations
    • Manage Orders
    • Categories
    • Basket Preferences
    • Customers
    • Upsell Cross Sell
    • Allowances
    • Customer Discounts
    • Order Preferences
    • Publish
    • Rebuild Solr
    • Email Setting
    • Store Preferences
    • Item Reviews
    • Tax Rates
    • Customer Segments
    • Store Currency
    • Product Data Spreadsheet
    • Export Orders
    • Sitemaps
    • Prices
    • Coupons
    • Bundles
    • Frequently Bought Items
    • Customer Properties
    • Store Properties
    • Multi Valued Properties
    • Gift Cards
    • Promotions
    • Manage Media
    • Gift Certificates
    • Manage Facets
    • Simple Add
    • Import and Export Using Spreadsheet
    • Customer Messages
    • Creating a Supplier Store
    • Approve Products
    • Import XLS
    • Business Reports
    • Item Properties

WordPress

  • Getting Started
  • Document Checklist
  • How to create pages in Wordpress?
  • How to create New Menus?
  • How to edit the Home Page?
  • How to change Favicon and Site Title?
  • How to edit Header and Footer?
  • How to change Header and Footer?
  • How to edit Shipping Policy, Refund Policy and Privacy Policy pages?
  • How to add Google Analytics and Microsoft Clarity
  • Using Existing WP vs Using Fresh WP
  • How to change Logo URL
  • How to create Blog Posts?
  • Blog Look And Layout
  • Netlify Redirects
  • Caching Issues
  • Plugins
Plugins
  • Perfmatters
  • Blocksy
  • Change SEO Title and Meta Descriptions with Yoast SEO
  • Staging and Backup with WPVivid
  • Stackable
  • Custom Javascript and CSS plugin
Common Issues
  • Does Not Use Passive Listeners
  • Enable Text Compression
  • Preload Largest Content Paint Image
  • Home
  • Docs
  • System Admin Pages
  • B2C -Overview
  • Marketing
  • A/B Split Tests

A/B Split Tests

Table of Contents
  • Introduction
  • Objectives
  • A/B Split Testing Overview
    • Split test process
  • Setting up an experiment in Google Analytics
  • Setting up the split test in Avetti Commerce
    • Add/Edit Split test case
    • Add/ Edit Personalization Details

Introduction #

This document will describe how to setup an A/B split test so that you know what content on your web page is most likely to convert visitors into customers more effectively. The whole purpose of doing a split test is to divide customers into groups and provide them with separate versions of the same page so that you can monitor which version is more successful when it comes to conversions.

Based on the reports generated, you would know Split test cases which can also be used to add complexity and experimentation to the content shown in a Content Area and to test the effectiveness of certain templates. It is a very powerful marketing tool.

Objectives #

By the end of this module, you will learn how to:

  1. Add and edit Split test cases and add groups.
  2. Add personalized content for user groups
  3. Set up an experiment on Google Analytics
  4. Integrate Google Analytics experiment with the Avetti split test feature so as to develop detailed reports.

A/B Split Testing Overview #

Split test process #

As for the purpose of experimentation, we create two or more user groups and separate templates for each group such that a user belonging to group A would see a different content on the same page when compared to a user belonging to group B. A user who has been exposed to a particular content will continue to see the same content when he comes back to the same page. As the experiment progresses through days, it would become evident that users of a particular group might be making more purchases or transactions compared to the other groups. From this you know what content is most appropriate to be put on your page so as to maximize conversions.

Setting up an experiment in Google Analytics #

The reason we chose to integrate with Google Analytics is because of the rich reports and graphs that it can generate. Remember that the experiment content set up and decision over what variation is to be shown is completely defined within the Avetti system. We are merely sending the experiment information to Google Analytics so that they can plot this information with graphs and custom reports.

At this point, we assume that you already have a Google Analytics account and that you have your pages being tracked.

In case you don’t, please follow these steps:

  1. Sign up for an account in Google Analytics (http://www.google.ca/analytics/).
  2. The Google tracking code has already been added to the footer template. So when you are asked to add the tracking code to your web page, simply go to the footer template and uncomment the tracking code section.
  3. Add your account number at the space provided. Once tracking is enabled you can verify this by clicking on admin | Tracking info. The status would stay ‘tracking code installed’ or ‘receiving data’.
  4. Once you have your tracking code setup and the pages are being tracked successfully you can start off an experiment. Go to the Reporting tab, select Behaviour on the left hand menu section and click Experiments.
  5. If it’s your first time on the experiments page, it will ask you for the page to test. Enter the URL of the page that you are conducting the test for and click start experimenting.

You need to go through the following four steps to complete the experiment setup process:

  1. Provide information:You need to enter details, such as name of the experiment, experiment objective and email notifications. Percentage traffic is determined by Avetti Commerce, changing the value here will not make any difference to the percentage of users exposed to the experiment. Under Objective for this experiment, select the metric that you want to improve, such as pageviews, transactions, or goal completions
  2. Setting up the variation pages: Since what we are doing is a content experiment that involves different contents but all in the same URL, this step is not very significant. Simply enter the URL for the page that we are testing as the original URL and enter any random value as the URL for the variation page. However, name the original page as ‘Group A’ and the variation page as ‘Group B’. This will help us later on as the experiment starts running.
  3. Set Experiment Codes:You would normally paste the experiment tracking code to the original page, but for now simply ignore this step, since we’ll be modifying the page and showing variations from our end. The experiment code provided by the Google web interface will not be used. Instead, get the Experiment ID which will be used later on to implement the experiment. You can find the experiment ID by clicking on the Manually insert the code button.

Note

  1. URLs are not applicable for this implementation. However, they are required to configure the experiment. You can ignore any errors or notifications such as “We could not find the URL you entered”
  2. The code validation errors can be ignored as the URLs provided during setup wont be used for the experiment. Changes are made on the same page with the same URL to show variations.

Setting up the split test in Avetti Commerce #

Now that you have setup a new experiment in Google Analytics and obtained the experiment code, it’s time to tell Google what to track. As mentioned earlier, what we are trying to do is a content based experiment. We basically create two separate contents so that one user group is exposed to one specific content while the other group will be shown the other content irrespective of the fact that they are all viewing the same page with same URL.

To set up the split test feature, we will be working with two separate sections on the Admin page.

  1. Add a new split tests case and edit it so as to add the user groups. This can be done under Marketing | A/B Split Tests.
  2. Specify what content is to be shown to a particular user group and when it is to be shown. This is done by editing personalization details under Templates | Catalog Personalization page.
Add/Edit Split test case #

To add a Split test case,

  1. Go to Marketing | A/B Split Tests. You will be taken to the Manage Split test case window.
  2. Click the Add button to open the Edit Split test case window and add a new case.
  3. Enter the details (Refer table 1).
  4. Click the Add group button to open the Edit split test group box.
  5. Enter the details (Refer table 2).
  6. Click Update and Ok button to save changes.
  7. Once you have finished creating your Split test case, you must add it to a Content Area on the Templates | Catalog Personalization page (Refer Add/Edit Personalization Details section)
FieldDescription
Case TypeSet the case type to ‘Google Analytics’ as we will be generating the report on Google Analytics.
Case CodeA code to represent the Split test case which is used for administrative purposes.
AvailableCheck this to ensure that the Split test case is active. If this is unchecked, the templates will not be shown.
Experiment IDEnter the experiment ID obtained while setting up the experiment in Google Analytics. It links the split test that we are setting up in Avetti with the experiment that has been set up in Google Analytics.
FieldsDescription
Group CodeThe code that represents a group in the Split test case. The first group (Group A) should have code set to 0. The Second group (Group B) code should be 1 and so on. This is a very crucial step as the Google reports rely majorly on these values.
Group NameEnter a brief name or description about the group.
Display MethodIn order to have a template displayed, it must first have a valid start/end date and be set to available.

There are three options for the Display Method, which determines how to cycle through multiple templates.

Priority by Date/Time: Always displays the valid template entry with the highest priority. The template entry will have options for start/end time, priority and available. If there are multiple templates available within the start/end dates, the template with the lowest number (same as highest priority) will be displayed. Under this Display Method, only one template will be shown for the given group until it’s not within the start/end dates, it is no longer available or its priorities are altered in such a way that the first template no longer has highest priority.

Sequence by Date/Time: Each time a new template is requested, the next template in the Content Area list will be loaded. It will find the current template sequence number (defined later) from a given customer’s cookie and display the template with the next sequence number.

Random by Date/Time: Templates are shown at random on each template request. Using this method, the templates can also be weighted so some templates have a higher chance of being shown over others; this is specified on the Define Split Test on Content Area page as Display Weight.
Display PercentageDisplay percentage (DP) determines how to distribute customers into split test groups.

Using this option, you can specify groups of equal sizes, or you can specify one group to be larger than the other. In most cases you may wish to keep the DP the same for each group, but if you require one test group to have a larger sample size than others, use this feature.

The math is as follows: Desired group size = (# of customers) * (DP of desired group) / ([DP of A] + [DP of B] + [DP of C] + …)

Example 1: 
There are groups A and B. If you wish group A to be the same size as group B, set the DP of each group to be the same (such as 100). In this case, if 20 customers are assigned to a group, each group would have 10 customers. The size of each group would be: 20 * 100 / (100 + 100) = 10

Example 2:
 There are groups A, B and C. If you wish group A to be the largest, while B and C are equal but smaller, set the DP of A to 100 and the DP of B & C to 50. Assume customer base is 20. The size of group A would be: 20 * 100 / (100 + 50 + 50) = 10 The size of each group B & C would be: 20 * 50 / (100 + 50 + 50) = 5
Time to DisplayDue to the customer’s ability to refresh the page many times in a short time, the templates shouldn’t switch on each page refresh. This feature will keep the current template active, through page refreshes, for as long as the Time to Display feature permits (in seconds, since the first time the page was loaded).
Add/ Edit Personalization Details #

This section is used to search for a template on any available skin.

  1. Refer Catalog Personalization for instructions on add/edit personalization details.
  2. In add/edit personalization page, Change the Content Type to “Split Test / Sequence of Templates”.
  3. Select the desired Split Test Case from the new dropdown below. If there are no Split Test Cases available, you must create one (which is covered in the above section).

You can add new templates once you have created this content group, as it can only be done in the Edit content group page. After the content group has been created, edit it and click on the new “Add Template” link next to the specified Split Test Case name.

On this page, the created groups will appear in their own box, where you can customize several things. Start by editing one of the Split Test Groups, and add a new template.

Enter the file name of an existing template that you wish to display for the specified group. Note: The contents of this template is what will be shown to users in this particular group once the experiment starts. If you have not created one, use the template manual to learn how to create a template. Create separate templates for each group. Example: As an example we have created 2 templates. Template A for group A and Template B for group B. The contents of the templates are to simply display a text highlighting which group you are in. We added the following script to Template A.

1| <h1>

2| You are in group A.

3| </h1>

What are your Feelings
Still stuck? How can we help?

How can we help?

Updated on October 27, 2022
Add-ons

Powered by BetterDocs

Table of Contents
  • Introduction
  • Objectives
  • A/B Split Testing Overview
    • Split test process
  • Setting up an experiment in Google Analytics
  • Setting up the split test in Avetti Commerce
    • Add/Edit Split test case
    • Add/ Edit Personalization Details

Copyright © 2022 Avetti.com Corporation.