Two people can disagree honestly about which display idea will sell more of the same style, and both can sound convincing. Debating the two ideas settles nothing on its own; running them both, in matched shops over the same period, produces an answer neither side can argue with.
This lesson gives you the method an agent uses to run a fair side-by-side test of two display ideas before recommending one to a client. You will apply it to Meridian Sourcing, choosing between a colour-blocked table and a mannequin-led window for the same Order A-style basic T-shirt, across two matched groups of a European buyer's own shops, because an agent representing more than one buyer needs a result it can defend with evidence, not preference.
You need: the idea of measuring a single display's own lift against a baseline, from Lesson 4.2, is useful background but not required.
By the end of this lesson you can:
Not covered here: deciding what to do with one display already running, based on its own lift alone (Lesson 4.3), or building the complete tracker across every display decision (Lesson 4.5). This lesson only teaches how to compare two ideas fairly before choosing one.
| Term | Meaning |
|---|---|
| Side-by-side test | Running two competing display ideas for the same style at the same time, in matched conditions, to compare their own results directly. |
| Matched shops | Two groups of shops chosen to be as similar as possible in size, location type, and recent sales, so a test's result reflects the display idea, not a difference between the shops. |
| Test variant | One of the two or more competing display designs being compared in a side-by-side test. |
| Statistical noise | Random day-to-day or shop-to-shop variation in sales that is not caused by either display idea, and that a fair test tries to average out. |
| Meaningful gap | A difference between two variants' own results large enough to be trusted as real, not just noise. |
| Control condition | Keeping every factor except the display idea itself the same between the two groups, so the comparison isolates that one variable. |
Testing one idea this month and a different idea next month cannot separate the ideas' own effects from whatever else changed between the two months, such as weather, a holiday, or a competitor's promotion. Running both ideas at the same time, in shops chosen to be as alike as possible beforehand, removes timing as a source of difference and leaves shop-to-shop variation as the main remaining noise, which matching also reduces.
| What can differ | The shapes you meet | Why it matters to you |
|---|---|---|
| Business model | A hybrid operation like an agent representing both Make to Order export clients and, through them, shops with their own domestic-style restock rhythm needs matching criteria that fit both: similar size and location type matter in either case, but a Make to Order client's shops also need a similar position in their own shared launch calendar. | Matching criteria must reflect what actually drives sales for that business model, not a generic checklist. |
| Number of shops available | An agent testing across a European buyer's larger shop network can match many pairs of shops and trust a smaller gap, since noise averages out more with more shops. A single-shop or very small test needs a much larger gap before trusting the result, since one unusual day can swing a small sample heavily. | Fewer shops in a test means a real effect has to be larger before it can be told apart from noise. |
| Market | An export brand's own shops in different countries add currency and local-holiday differences on top of shop-to-shop variation, so matching should also consider country or region where possible. A domestic chain testing within one country avoids that extra layer of difference. | Cross-border testing needs an extra matching factor that a single-country test does not. |
| Product family | A basic T-shirt or apparel style shows a lift or gap within one to two weeks, given how often it is repurchased or seen. A bedding or home-textile style, bought less often, may need a longer test period to collect enough sales to trust a comparison. | The test period length should fit how often the product family is normally bought, not a single fixed rule for every style. |
| Number of ideas tested | Testing two ideas splits available shops into two groups; testing three or four ideas at once splits the same shops further, so each group's own result carries more noise unless the agent has enough shops to keep each group a reasonable size. | More ideas tested at once need proportionally more shops to keep each comparison reliable. |
Meridian Sourcing, an agent representing two European buyers, is asked to recommend one display idea for a new basic T-shirt style, matching Order A's own build, across one buyer's shop network. Two ideas are on the table: a colour-blocked table display, and a mannequin-led window display. Rather than pick one on opinion, Meridian Sourcing splits the buyer's comparable shops into two matched groups of similar size and location type, and runs one idea in each group for the same two-week period.
Nothing else differs between the groups during the test: the same style, the same starting stock level, the same two-week window, and no price change in either group.
The worked numberAn 8-percentage-point gap, measured over matched shops in the same period, is a meaningful gap rather than ordinary noise, since both groups faced the same timing and were chosen to be alike beforehand. Meridian Sourcing can recommend the mannequin-led window to the buyer with evidence behind it, rather than a preference stated without a test.
A colour-blocked table tested during a shop's peak weekend traffic will often show a higher raw lift than a mannequin window tested during a slow midweek period, regardless of which idea actually works better, because the timing itself, not the idea, drove most of the difference. The cost of this mistake is not the test itself; it is a wrong recommendation rolled out across an entire shop network on the strength of a comparison that was never fair to begin with, undoing any value the test was meant to add.
When two people disagree about which display idea will sell more, do not argue; test both at the same time, in shops chosen to be as alike as possible beforehand. Run each idea in its own matched group for the same period, then measure each group's own lift the same way. If one idea's lift is clearly bigger, by more than could plausibly be chance, recommend it with that evidence. If the gap is small, treat the two ideas as roughly equal rather than picking a winner from noise.
The mannequin-led window's 17% lift looks strong against the colour-blocked table's 9%. Partway through reviewing the result, Meridian Sourcing discovers that three of the mannequin-led window's own shops ran an unrelated loyalty-card sign-up promotion during the same two-week test period, which the colour-blocked table's group did not have.
Should Meridian Sourcing recommend the mannequin-led window on the strength of the full 8-point gap? Decide before opening the answer.
No. With half of the mannequin-led window's own shops running an unrelated promotion during the test, the 8-point gap cannot be trusted as the display idea's own effect alone; Meridian Sourcing should re-test with cleaner matched conditions before recommending either idea.
1. Half of one group's shops carried an extra, unmatched factor, the loyalty-card promotion, which breaks the control condition the method depends on. 2. The colour-blocked table's group had no equivalent promotion, so the two groups are no longer comparable on this point, regardless of how the display ideas themselves perform. 3. A recommendation based on this result risks crediting the mannequin window for a lift the promotion actually caused, in whole or in part, in exactly the shops where it ran.
The method's own matching step exists to prevent this; when a control condition breaks partway through a test, the honest response is to flag it and re-test, not to report the result as if the break never happened.
Order A's T-shirt carries a shop price of USD 19.90 per piece (docs/02 section 7). If Meridian Sourcing wrongly credited the full 17% lift to the mannequin window for a 40-shop rollout, but the true lift is closer to 10%, the shortfall becomes:
A bad rollout does not fail visibly on day one; it quietly underperforms every week against a forecast built on a gap that was never real, a far larger cost than the two extra weeks a clean re-test would take.
Use the T22-C04 Display-performance tracker workbook, "Side-by-side test log" sheet. For each test, record the two variants, the matched shop groups, the test period, each group's own lift, and the gap, so a past test can be checked before running a new one. Start with Meridian Sourcing's example above, then add your own tests.
[Download the Display-performance tracker workbook](../../templates/T22-C04-display-performance-tracker.xlsx)
1. Why does testing two display ideas at the same time, in matched shops, give a more reliable comparison than testing them one after another in different months?
2. Two display ideas are tested in matched shops over the same two weeks; Idea A shows a 6% lift and Idea B shows a 7% lift. What should the agent conclude?
3. A test compares two display ideas, but one group's shops are, on average, in busier locations with more foot traffic than the other group's shops. What does this do to the test?
| EN | AR | FR | TR | Meaning |
|---|---|---|---|---|
| Side-by-side test | اختبار جنبًا إلى جنب | Test côte à côte | Yan yana test | Running two competing display ideas for the same style at the same time, in matched conditions, to compare their own results directly. |
| Matched shops | متاجر متطابقة | Magasins appariés | Eşleştirilmiş mağazalar | Two groups of shops chosen to be as similar as possible in size, location type, and recent sales, so a test's result reflects the display idea, not a difference between the shops. |
| Test variant | متغيّر الاختبار | Variante testée | Test varyantı | One of the two or more competing display designs being compared in a side-by-side test. |
| Statistical noise | التشويش الإحصائي | Bruit statistique | İstatistiksel gürültü | Random day-to-day or shop-to-shop variation in sales that is not caused by either display idea, and that a fair test tries to average out. |
| Meaningful gap | فارق ذو دلالة | Écart significatif | Anlamlı fark | A difference between two variants' own results large enough to be trusted as real, not just noise. |
| Control condition | شرط الضبط | Condition de contrôle | Kontrol koşulu | Keeping every factor except the display idea itself the same between the two groups, so the comparison isolates that one variable. |
Next you will bring everything from this course together into one complete tracker, covering every display decision from the first brief through the build, the measurement, and the final choice. Back to the course map