Adobe Analytics Plus ChatGPT for Ideation: Part 1 – A/B Testing
Sometimes, the scope of experimentation can be overwhelming, especially if you’re in a small team or just starting out.
How do you generate relevant test ideas for your organisation? What are the best practices?
In this two-part edition of the ‘Adobe Plus’ series, I’m going to look at how ChatGPT can be a useful tool to give you a kickstart and a fresh perspective on ideation. In Part 1, I’m going to look at its application in A/B test ideation.
What’s the catch?
- ChatGPT is only as good as the context you give it.
- You know your organisation’s data and audience best.
- Use ChatGPT’s suggestions in conjunction with your own data about your customers to validate and make decisions on what to test.
It goes without saying, but don’t upload sensitive or personal data to ChatGPT.
Use ChatGPT to generate A/B test ideas for you
Start by giving ChatGPT as much context as possible – tell it at least your industry and goals, but the more context, the better!
Prompt: Act like a conversion rate optimisation manager for a retail company with a website that sells shoes.
You can ask it for best practices for specific pages or websites in your industry.
Prompt: What are the necessary elements of a product page on a website that sells shoes?
Example responses:
- Size and Fit Information: Provide guidance on sizing, including a size chart if applicable, to help customers choose the right size. Include information about the fit of the shoe (e.g., true to size, narrow, wide) to manage expectations.
- Product Reviews and Ratings: Customer reviews and ratings provide social proof and help build trust. Display reviews and ratings prominently on the product page to help customers make informed decisions.
- Return Policy: Clearly communicate the return policy, including information about returns, exchanges, and refunds. A transparent and customer-friendly return policy can help alleviate concerns and increase confidence in making a purchase.
At this point, you could cross-check the suggested best practices against your site. What’s missing? What’s present, but could be improved?
ChatGPT can also give you A/B test ideas for a specific page or user journey:
Prompt: Generate 10 website experimentation ideas to A/B test on a product page, aiming to increase purchases.
Example responses:
- Product Description Optimization: Test variations of product descriptions, including length, tone, and formatting. Focus on highlighting key features, benefits, and unique selling points to better persuade potential customers.
- Social Proof Integration: A/B test the placement and format of social proof elements such as customer reviews, ratings, and testimonials. Determine whether displaying this information prominently near the “Add to Cart” button influences purchase decisions positively.
- Price Display: Experiment with different ways of displaying pricing information, such as including sale prices, discounts, or installment options. Test whether showing the price upfront or after selecting product options leads to higher conversion rates.
Prompt: Generate 10 website experimentation ideas to A/B test throughout the checkout funnel, aiming to increase purchases.
Example responses:
- Guest Checkout vs. Account Creation: Test whether offering a guest checkout option without requiring account creation leads to higher conversion rates compared to mandatory account creation. This reduces friction for users who are in a hurry or reluctant to create an account.
- Prominent Security Seals and Trust Badges: Test different placements and designs of security seals and trust badges throughout the checkout process to build trust and reassure users about the security of their personal information.
- Shipping Cost Transparency: Test displaying shipping costs upfront early in the checkout process, preferably on the product page or cart page, to avoid surprises and potential abandonment at the last minute.
If you need some direction, this can provide a good starting point for analysing user behaviour on your website to inform A/B test ideas.
- Do you have analytics tracking in place to measure the engagement and value of the features on your site? Is it fit for purpose? If not, can this be updated, or obtained from other tools?
- Which site features have the most and least engagement? What does this say about what users are interested in?
- Do any interactions or user journeys suggest frustration or confusion (e.g. looping behaviour or repeated clicks/views)?
You can then ask ChatGPT for more specific ideas based on your Adobe Analytics data, including channels, audiences and KPIs.
Prompt: There is a high bounce rate on the product page and visitors are going back and forth between the product page and the checkout page on the website. What could be causing this and how could it be addressed?
Example responses:
- Unclear Product Information: Visitors may be bouncing because they cannot find the information they need about the product. This could include insufficient product descriptions, unclear sizing information, or lack of detailed images.
- Poor User Experience (UX): The product page may have usability issues or be difficult to navigate, leading visitors to abandon the page prematurely. This could include confusing layout, unclear navigation paths, or intrusive pop-ups.
- Lack of Trust Signals: Visitors may hesitate to proceed to checkout if they perceive the website as untrustworthy. This could be due to missing trust signals such as customer reviews, security badges, or a transparent refund policy.
To address these issues and reduce bounce rate while encouraging visitors to proceed to checkout, consider the following strategies:
Optimize Product Page Content: Ensure that product descriptions are clear, informative, and address common questions or concerns. Include high-quality images from different angles and provide detailed sizing information to help visitors make informed purchasing decisions.
- Enhance User Experience: Conduct usability testing to identify and address any UX issues on the product page. Make it easy for visitors to find what they’re looking for by optimizing navigation, streamlining the checkout process, and removing any unnecessary distractions.
- Build Trust and Credibility: Display trust signals prominently on the product page, such as customer reviews, ratings, and security badges. Clearly communicate your website’s security measures and return policy to reassure visitors and build trust.
Using GPT-4
At the time of writing, the free version of ChatGPT uses model version GPT-3.5. GPT-4 is used in the paid version of ChatGPT, which is OpenAI’s more advanced system.
GPT-4 can browse URLs and analyse screenshots, so you can point it to specific URLs and provide screenshots as additional sources of information. For example:
Prompt: Act like a conversion rate optimisation manager for a retail company that has a website that sells shoes: https://www.toms.com (please research the site to get up-to-date context). Also, use your knowledge of the provided screenshots of the website product pages. Generate 10 website experimentation ideas to A/B test on these pages, aiming to increase purchases.
Our findings were:
- Using GPT-4, providing screenshots along with URLs had better results than just providing URLs alone.
- There was always overlap between the results provided by ChatGPT and GPT-4.
- ChatGPT tended to provide broader best practice & tactical ideas, vs GPT-4’s more specific & strategic ones. However, you may be able to get round this in ChatGPT by providing more context and detailed prompts!
In summary…
ChatGPT can be used as a great starting point for generating ideas for testing – it’s then up to you to apply knowledge of your organisation to create a truly data-driven roadmap.
To find out more about our booster programmes for data-driven conversion rate optimisation, please contact us.