An AI tool that drafts a clear email or summarises a long thread well can, in the very next request, state a wrong number, a wrong date or an invented detail with exactly the same confident tone. This is not a random flaw that will simply disappear with a better tool; it follows directly from how the tool produces answers in the first place, and it shows up on domestic stock work just as it does on export orders.
This lesson explains the underlying reason an AI tool makes these kinds of mistakes, using Delta Garments' own domestic stock work for Cedar & Co. as the case. Understanding the reason, rather than memorising today's list of known errors, is what lets you recognise a new kind of mistake tomorrow, on a tool you have not used before.
You need: the working idea of what an AI tool does from lesson 1.1 and the task-matching idea from lesson 1.2 are useful preparation but not required.
By the end of this lesson you can:
Not covered here: how to fix or retrain an AI tool, or a technical explanation of the tool's internal design. Lesson 1.4 covers the practical method for checking an AI-generated answer.
| Term | Meaning |
|---|---|
| Confident wrong answer | An AI-generated response stated in a fluent, assured tone despite being factually incorrect. |
| Invented detail | A specific fact, number or name an AI tool produces that was never supplied to it and does not exist in reality. |
| Pattern-based answer | A response built from what commonly appears in similar text, rather than from the actual facts of your own situation. |
| Routine-task blindness | The tendency to stop checking an AI-generated output carefully once a task starts to feel repetitive and familiar. |
| Source document | The original record that holds the true version of a fact, which an AI-generated output should be checked against. |
Because an AI tool works by predicting likely, well-formed text, it does not have a built-in way to say "I do not know this specific fact about your business" in the way a person naturally would. Instead, when asked something outside what it was given, it tends to produce the most plausible-sounding answer based on common patterns in similar text, delivered in the same confident tone as a correct one. This is the direct cause of most of the mistakes an AI tool makes: not carelessness, but the fact that fluent, pattern-based text and factually correct text look identical on the page.
| What can differ | The shapes you will meet | Why it matters to you |
|---|---|---|
| Domestic replenishment under Make to Stock, factory side | Delta Garments' own planning team asks an AI tool to draft a weekly reorder summary for Cedar & Co.'s stock programme, week after week. | A task repeated many times invites routine-task blindness: the tenth summary is checked less carefully than the first, even though the risk of a wrong figure has not fallen. |
| Export order under Make to Order | A one-off buyer query about Order A's specification is checked closely because it is unfamiliar and feels higher stakes. | A task that feels novel gets more scrutiny by habit, which is one reason mistakes on routine domestic work are more likely to slip through unnoticed. |
| Hybrid order handled through an agent | An agent asking an AI tool to fill a gap in incomplete buyer instructions risks receiving a plausible but invented requirement. | A gap in supplied information is exactly the condition under which a pattern-based, invented answer is most likely to appear. |
| A new user compared with an experienced user | A new user may not yet know which of Cedar & Co.'s own figures are fixed and checkable, so an invented number can pass unnoticed. | Recognising a source document to check against depends on knowing which facts in your own business are fixed; this knowledge is what makes the mistake catchable. |
| Different product families | A knit-to-garment or terry order's own gauge, GSM or pile-height figures can each be invented just as easily as a T-shirt's fabric weight if the fact is not supplied. | The underlying cause is the same across every product family: an unsupplied fact invites a plausible, unverified guess. |
Cedar & Co., a domestic brand with 12 shops, buys from Delta Garments on a Make to Stock basis, reordering against its own sales forecast. Delta Garments' figures for a specific Cedar & Co. reorder are not part of the master figures used elsewhere in this course; for this lesson, the same fabric facts already fixed for Order A (single jersey, 180 GSM, Ne 30/1) are reused as the known, checkable reference the planning assistant should have compared the AI-generated summary against.
The planning assistant asks an AI tool to draft this week's reorder summary but does not paste in the actual current stock sheet. The tool returns a fluent, well-organised summary that states a plausible-sounding fabric specification. Because the summary reads clearly and the task feels routine after several weeks of doing it, the assistant sends it without comparing the stated specification against Delta Garments' own fixed fabric facts.
The worked numberA common mistake is assuming that because an AI tool has produced a correct-looking summary many times before, the next one needs less checking. The task itself has not changed; only the person's own attention to it has. The cost of that assumption is not visible on the ninth or tenth correct summary, only on the one time a gap in supplied information lets a plausible, invented detail through unnoticed.
An AI tool cannot tell you it does not know something the way a person can. Ask it a question it has the real answer to, and it answers well. Ask it a question it does not have the real answer to, and it still answers, just as smoothly, filling the gap with what commonly appears in similar text. The two answers read exactly the same on the page: the same clear sentences, the same confident tone, the same plausible level of detail. Only checking the actual fact against a real record, such as Delta Garments' own fabric sheet, ever tells the two kinds of answer apart from each other.
The AI-generated reorder summary the planning assistant sent to Cedar & Co. states the reorder fabric as "190 GSM single jersey," a figure the assistant never typed into the tool and never checked against Delta Garments' own fabric record before sending.
Should Delta Garments confirm 190 GSM to Cedar & Co. because that is what the AI-generated summary stated? Decide before opening the answer.
No. Delta Garments must confirm 180 GSM, its own actual fixed fabric reference, and correct the sent summary.
1. The 190 GSM figure was never supplied to the AI tool by the planning assistant, so it is a plausible, pattern-based number, not a checked fact from Delta Garments' own records. 2. 180 GSM is the fabric weight actually fixed and used in Delta Garments' own production, the same weight already confirmed for Order A, and is the only figure with a real source behind it. 3. Confirming the wrong figure to Cedar & Co. would cost far more to correct after fabric or costing decisions are made on it than the few minutes a check would have taken before sending.
The AI tool did not make an error in the sense of malfunctioning; it filled a gap it was never given the real answer to. The mistake was sending the result before comparing it against Delta Garments' own known figure.
Order A's fabric reference used here for comparison is fixed at:
A mistake caught before sending costs only the few minutes of the check. A mistake caught after it reaches a buyer, such as Cedar & Co. querying a mismatched specification, costs the time to trace the error, correct the record, and rebuild the buyer's confidence in the factory's own routine updates, as shown in the 42 extra minutes worked out above. The cost rises the further the wrong fact travels before someone catches it.
Use the T18-C01 AI use worksheet. In the Risk log sheet, record each AI-generated output where part of the answer was not built from a fact you supplied, and note whether you compared it against a source document before using it. Start with the Cedar & Co. reorder example, then repeat the exercise for a task from your own week.
[Download the AI use worksheet](../../templates/T18-C01-ai-use-worksheet.xlsx)
1. Why did the AI tool state '190 GSM' for Cedar & Co.'s reorder when Delta Garments' own fixed fabric reference is 180 GSM?
2. A domestic brand's planning team has used an AI tool to draft the same weekly reorder summary for six months without a single caught mistake. What does this history tell them about the risk of an unchecked mistake next week?
| EN | AR | FR | TR | Meaning |
|---|---|---|---|---|
| Confident wrong answer | إجابة خاطئة واثقة | Réponse erronée assurée | Kendinden emin yanlış yanıt | An AI-generated response stated in a fluent, assured tone despite being factually incorrect. |
| Invented detail | تفصيلة مختلقة | Détail inventé | Uydurulmuş ayrıntı | A specific fact, number or name an AI tool produces that was never supplied to it and does not exist in reality. |
| Pattern-based answer | إجابة مبنية على نمط | Réponse fondée sur un motif | Örüntü tabanlı yanıt | A response built from what commonly appears in similar text, rather than from the actual facts of your own situation. |
| Routine-task blindness | تجاهل المهمة الروتينية | Aveuglement lié à la routine | Rutin göreve karşı dikkatsizlik | The tendency to stop checking an AI-generated output carefully once a task starts to feel repetitive and familiar. |
| Source document | المستند المصدر | Document source | Kaynak belge | The original record that holds the true version of a fact, which an AI-generated output should be checked against. |
Next you will learn a practical method for checking an AI-generated answer before relying on it, seen through Meridian Sourcing's own use of AI on a hybrid order. Back to the course map