Sara, a buyer at Northline Apparel, does not see which AI tools a factory's staff use day to day. She sees the results: a reply that answers a question she did not ask, a specification that is subtly different from the one she approved, a delivery date that changes without a clear reason. When a factory or buying office has no rule for how AI tools are used, these results start appearing more often, and Sara has no way to know which of them came from a careless AI-drafted output that no one checked.
This lesson mainly teaches the decision flow buyers and factories both need: what actually goes wrong when AI use is left ungoverned, so the policy from Lesson 5.1 is not just a formality but a response to real, specific risks. Without seeing these risks clearly, a business tends to treat an AI-use policy as paperwork rather than as the thing standing between an AI tool's mistake and a customer.
You need: Lesson 5.1's reasoning for why an AI-use policy exists. This lesson goes further by naming the specific risks a missing policy allows.
By the end of this lesson you can:
Not covered here: how to evaluate a specific AI tool before adoption (Lesson 5.3) and how to assign accountability once something has already gone wrong (Lesson 5.4).
| Term | Meaning |
|---|---|
| Ungoverned AI use | AI use with no written policy, no information rule and no human review step in place. |
| Silent error | A mistake in an AI-drafted output that reads as confident and correct, so no one notices it before it is sent or acted on. |
| Error at scale | The same mistake repeated automatically across many documents, orders or messages because an AI tool produced it once and it was copied or reused. |
| Leaked specification | A technical or commercial detail from a confirmed order shared outside the people who are meant to see it. |
| Broken commitment | A promise made to a buyer or a factory, in writing, that turns out not to match the real facts. |
Each of these three risks follows directly from what an AI tool actually does. It can retain and reuse whatever it is given, which is how a specification leaks. It produces confident, fluent text whether or not the underlying fact is right, which is how a silent error slips past a reader who is not checking carefully. And because it can produce the same kind of output instantly and repeatedly, one uncaught mistake in its method can appear in many documents before anyone notices the pattern.
None of these three risks requires the AI tool to be unusually poor; they follow from ordinary AI behaviour meeting a business process that has no policy, no information rule and no review step to catch them.
| What can differ | The shapes you will meet | Why it matters to you |
|---|---|---|
| Export order under Make to Order, buyer's side | A buyer receives an AI-drafted reply from a factory that answers confidently but does not match the confirmed production plan. | The buyer has no way to know the reply was AI-drafted or unchecked; the cost of trusting it falls on the buyer's own planning. |
| Domestic replenishment under Make to Stock | A domestic brand's own planning team lets an AI tool draft a reorder recommendation from sales history with no review. | A wrong reorder quantity ties up real cash in stock that will not sell, discovered only when the stock does not move. |
| Hybrid order through a mill | A mill drafts a material specification with an AI tool and reuses the same draft across two buyers' hybrid orders without checking either one against its own buyer's approval. | One silent error in the shared draft becomes an error at scale, repeated identically across more than one order at once. |
| Buying office holding more than one buyer's information | Staff paste one buyer's confirmed price into an AI tool while asking a general question, without meaning to expose it. | A leaked specification or price is a breach of confidentiality even when no one intended to share it; intent does not change the exposure. |
| Different product families | Socks, knit-to-garment, terry and home-textiles orders all carry the same three risks; only the specific document at risk (a spec line, a fault log, a border detail) changes. | The three risks are general to any AI use on confirmed order information, not specific to one product type. |
Northline Apparel has confirmed Order A with Delta Garments: 15,000 T-shirts, single jersey 180 GSM, Ne 30/1, agreed FOB USD 3.20 per piece, total order value USD 48,000. Sara asks Delta Garments to confirm the shipment is still on track for the agreed week. She receives a fluent, confident reply confirming the original week, signed by a member of Delta Garments' merchandising team.
What Sara does not know is that the reply was drafted by an AI tool from an older version of the production schedule, because the staff member had not updated the information given to the tool after a two-day delay was recorded that same morning. No one at Delta Garments reviewed the reply against the current schedule before it was sent, because no policy required it.
Sara plans her own warehouse receiving schedule around the confirmed week. When the shipment arrives two days late, she has no reason to have expected it, because the reply she received gave her no signal that anything had changed. The cost of the ungoverned AI use lands on Sara's side of the order, not on Delta Garments' side where the mistake was made.
The worked numberA 2-day silent error is small next to a 2-week planning window, so it may not cost the whole order's value in this case; but the same ungoverned process could just as easily hide a longer delay, a wrong quantity, or a wrong specification detail, each capable of putting the full USD 48,000 order value at risk if it reaches Sara unreviewed. The size of the risk is set by what is at stake in the order, not by how small this particular error happened to be.
An AI tool that drafts fluent, confident replies feels reliable, which is exactly why a silent error inside it is dangerous: the fluency of the output has no relationship to whether the underlying fact is correct. Judging risk this way misses the real question, which is what confirmed information the tool touches and who reads its output without checking it.
The better rule is simple: judge an AI tool's risk by what confirmed business information it touches and who receives its output unchecked, not by how confident its answers sound.
An AI tool that has no rule around it is like a fast, confident messenger who will repeat back whatever it half-remembers, in a calm and convincing voice, without knowing whether it is right. If that messenger carries your confirmed price, your confirmed date or your confirmed specification without anyone checking what it says first, a small mistake can reach a buyer or a factory as if it were fact, and it can reach several buyers or several orders in the time it takes to reuse the same draft.
The fix is not to stop using the tool; it is to know exactly where its output reaches someone who will act on it, and to put a person there before it does.
A member of Delta Garments' team uses an AI tool to summarise a long internal inspection report into a short update for Sara, ahead of a scheduled call. The AI-generated summary reports the shipment as passed with no issues. The original inspection report, which the team member did not re-read after generating the summary, in fact records one fault type above the agreed acceptable quality limit on a small sample of cartons.
Should Sara authorise payment release based on the summary as sent? Decide before opening the answer.
No. Payment release is held until the original inspection report is checked directly against the fault limit.
1. The AI-generated summary omitted a fault the original report clearly recorded, which shows the summary cannot be trusted as a substitute for the report itself on a payment decision. 2. No one compared the summary against the original report before it was sent, which is the missing control, not a reason to trust the summary this one time. 3. A payment release decided on a wrong summary would need to be reversed or disputed once the fault is found, which costs far more time and trust than checking the original report now.
The cost of releasing payment on the wrong summary is not only the fault itself; it is that Sara would have released payment on a document that did not reflect the real inspection result, which weakens every future summary Delta Garments sends her.
Order A's fixed total order value shows what is already at stake before any AI-related risk is added:
An ungoverned AI tool rarely destroys this value in one visible event. Its real cost is a leaked specification that undermines a confidentiality agreement, a silent error that reaches a buyer as if it were fact, or a mistake repeated across several documents before anyone notices the pattern. Each of these is harder to catch than an obvious mistake, and each one erodes the trust a buyer places in every future reply, quote or report the business sends, well beyond the value of the one order where it happened.
The cheapest point to prevent this cost is exactly where Lesson 5.1's policy places it: before the AI-drafted output leaves the business unreviewed.
Use the T18-C05 AI governance and risk checklist workbook. In the AI risk points sheet, list each place in the business where AI-drafted output reaches someone outside the team that drafted it, what kind of information is at risk there, and whether a named reviewer currently checks it. Start with Delta Garments' own reply and summary risk points on Order A, then repeat the exercise for your own business.
[Download the AI governance and risk checklist workbook](../../templates/T18-C05-ai-governance-and-risk-checklist.xlsx)
1. Why is a silent, confident-sounding error in an AI-drafted reply more dangerous than an obviously wrong one?
2. Order B (3,000 jeans, agreed target FOB USD 9.80 per piece) has an AI-drafted specification reused, unreviewed, across two hybrid orders that both use it. One error in the draft is repeated in both orders. What kind of risk is this?
| EN | AR | FR | TR | Meaning |
|---|---|---|---|---|
| Ungoverned AI use | استخدام غير محكوم للذكاء الاصطناعي | Usage non gouverné de l'IA | Yönetişimsiz yapay zekâ kullanımı | AI use with no written policy, no information rule and no human review step in place. |
| Silent error | خطأ صامت | Erreur silencieuse | Sessiz hata | A mistake in an AI-drafted output that reads as confident and correct, so no one notices it before it is sent or acted on. |
| Error at scale | خطأ على نطاق واسع | Erreur à grande échelle | Ölçekte hata | The same mistake repeated automatically across many documents, orders or messages because an AI tool produced it once and it was copied or reused. |
| Leaked specification | مواصفة مسربة | Spécification divulguée | Sızdırılmış spesifikasyon | A technical or commercial detail from a confirmed order shared outside the people who are meant to see it. |
| Broken commitment | التزام غير محقق | Engagement non tenu | Yerine getirilmeyen taahhüt | A promise made to a buyer or a factory, in writing, that turns out not to match the real facts. |
Next you will learn what to check, before you adopt any AI tool, so it does not create the same risks this lesson has just shown you. Back to the course map