A Shopify skincare brand sells its best-selling serum at $38. The founder suspects $42 would sell almost as well. If she is right, every month at $38 leaves margin on the table. If she is wrong, a price rise quietly cuts orders and nobody can say why.
A price test answers that question with data instead of instinct. It is also the test type with the most ways to go wrong in front of customers. Layout and copy experiments, like the ecommerce A/B tests most stores start with, change how a store looks. A price test changes what a customer pays, and customers notice when that feels unfair.
Why Is Price Testing Riskier Than Other A/B Tests?
In September 2000, Amazon ran a five-day test on 68 DVD titles, with discounts that varied at random between 20 and 40 percent. Shoppers compared notes, found they had paid different prices for the same discs, and complained publicly. Amazon refunded 6,896 customers an average of $3.10 each and changed its policy so that anyone who bought a test item would get the lowest test price.
Amazon’s method was sound. It randomized the discounts and did not use customer data to set them. What it had not planned for was the moment customers found out.
That is the difference between a price test and a headline test. A shopper who sees a different hero image never knows. A shopper who sees a different price can screenshot it, post it, or email support. Every price test needs a plan for that customer before it goes live.
What Should You Measure in a Price Test?
Conversion rate is the wrong primary metric for a price test. A higher price almost always lowers conversion a little. The question is whether the extra margin per order more than covers the lost orders.
Use revenue per visitor as the primary metric, and gross profit per visitor if you know your unit costs. Here is how the serum example plays out per 1,000 visitors, with a unit cost of $10:
| Metric | $38 (control) | $42 (variant) | Change |
| Conversion rate | 3.0% | 2.8% | -6.7% |
| Orders | 30 | 28 | -6.7% |
| Revenue | $1,140 | $1,176 | +3.2% |
| Gross profit | $840 | $896 | +6.7% |
A team watching conversion rate would call the $42 price a loser. A team watching profit per visitor would ship it. Pick the metric before the test starts, write it down, and judge the result on that metric alone.
How Do You Run a Price Test on Shopify?
A clean price test has five parts. Skip one and the result stops meaning much.
1. Pick one product or a small group with steady traffic. Best sellers work well because they reach a readable sample faster. Avoid products in a launch window or a sale.
2. Test two prices, not five. Each extra price point splits your traffic further and stretches the test.
3. Split traffic at random. Visitors should not be sorted by device, location or traffic source, or the groups will differ before the test begins.
4. Keep the price consistent everywhere a visitor sees it. Collection pages, product pages, the cart and checkout must all show the same assigned price.
5. Decide the primary metric and run length up front. Write both down before launch.
Doing this by hand in Shopify usually means duplicating products or redirecting part of the traffic to a copy of the page. That splits reviews and inventory across two products and creates near-duplicate pages for search engines to sort out. A dedicated Shopify A/B testing app such as Elevate changes the price on the original product page instead, splits traffic between the two price points and reports revenue per visitor for each. Whichever tool you use, test the full purchase path yourself in both variants before real customers see it.
How Do You Keep Customer Trust During a Price Test?
Price tests that turn into support tickets usually skip the preparation below.
• Decide your price-match policy in advance. If a customer finds the lower test price and asks for it, give it to them. A few refunds are a small cost against a clean result and a customer who stays.
• Brief your support team. They should know which products are in a test, what the two prices are, and what they are allowed to offer.
• Leave existing agreements alone. Active subscribers, wholesale accounts and loyalty members on a fixed price should not be pulled into a test.
• Keep the gap reasonable. A 5 to 15 percent difference is often large enough to measure on a best seller. A 40 percent gap invites screenshots.
• Be careful with compare-at prices. Testing a crossed-out “was” price is a discount claim, and many markets regulate how those are shown. The EU, for example, has strict rules on reference prices for announced discounts, so check local rules first.
• Avoid big promotional periods. Black Friday traffic behaves differently, and shoppers compare prices more closely when they expect deals.
How Long Should a Price Test Run?
Long enough to reach a usable sample, and small differences need large samples. If a product converts at 3.0 percent and you want to detect a 10 percent relative change, to 3.3 percent, you need roughly 53,000 visitors per price point at 95 percent confidence and 80 percent power.
Two rules keep the timing honest:
• Run in full weeks. Weekday and weekend shoppers often behave differently. Two full weeks is a sensible minimum, even if the numbers look settled sooner.
• Do not stop on a good day. Checking results every morning and ending the test the first time the variant leads inflates false positives. Set the run length or stopping rule before launch and hold to it.
If a product cannot reach the sample in four to six weeks, test a bigger price gap, test a group of products together, or move the test to a higher-traffic product.
What Should You Do After a Price Test Ends?
Once the winning price is live, three checks make the decision safer.
1. Watch the next 30 to 60 days. A higher price can lift first-order profit and still hurt repeat purchases or raise refund rates. Compare repeat rate and refunds for the test cohort against the period before.
2. Honor your promises. If you committed to giving test buyers the lower price, issue those refunds as soon as the test closes.
3. Record the result. Note the product, both prices, dates, traffic, the primary metric and the outcome. Price sensitivity changes with seasons and competitors, so a result from spring may not hold by the holidays.
Is Price Testing Worth It for Your Store?
For a store with steady traffic on its best sellers, usually yes. Price is the one lever that changes margin on every order, and a well-run test turns a guess into a number. Start with one product, two prices and one metric chosen in advance, and plan for the customer who notices before the test goes live.
