The experiments came out of the data
Before shipping anything, we analysed 20M+ sessions and 100M+ events in BigQuery across web and app. Each experiment came out of that analysis.
The analysis stood on a data warehouse I architected about five years ago: all schemas, data models and field definitions. It is still the company's only source of truth.
Navigation between products
The homepage
Traffic channels
Instagram and Facebook in-app browsers
The app
From events to launches
Data warehouse
source of truthSchemas, data models and field definitions, architected in-house.
Session and event analysis
BigQueryA/B-tested launches
experimentsRevenue per session
north starThe measure the programme moved.
40+ launches across the funnel
The programme shipped 40+ A/B-tested launches across the product page, checkout and payments, including many offer tests.
Product page
A/B-tested launches on the PDP.
Checkout
A/B-tested launches in checkout.
Payments
A/B-tested launches in payments.
Offers
From basic discounts to buy-X-get-Y-free.
Offers that sell without eroding margin
We tested a wide range of offers, from basic discounts to buy-X-get-Y-free, and introduced the free-products proposition. It has been the biggest sales driver for 4 years.
Offers at that scale need controls. The first-time welcome offer was spilling over to repeat users, and discount spend rose as offers scaled, so we built two.
Free products
The biggest sales driver for 4 years.
Coupon personalisation
The right coupons for first-time, churned and repeat users. First-time conversion rose slightly, and welcome-offer use by repeat users fell.
Cart-level discount capping
One coupon code, with a dynamic discount set by cart value and the cart's CM1. It saved millions in CM1.
Discounts stayed at a stable, healthy share of revenue.
What changed
Revenue per session rose 69% in 6 months. Discounts held at a stable, healthy share of revenue, and the CM1 profile improved overall.