Lesson 0.3Lesson 0.3 · Foundations of AI-Assisted Design
What AI Is Good & Bad At
The mental model that decides everything downstream - AI is superb at speed, volume, drafting, summarising and pattern-finding, and unreliable at truth, judgement, real context, accountability and novelty
AI will confidently give you a brilliant first draft and a completely invented building code in the same breath - the skill is knowing which is which.
Lesson 0.1 sketched the line between what AI is good at and weak at. This lesson turns that sketch into a working instrument - one you consult, half-consciously, every time you consider handing a task to a machine.
The most important thing to understand is why the line falls where it does. AI models do not know things the way you do; they produce statistically plausible continuations of a pattern. That single fact explains both the magic and the menace: it is why they draft, summarise and generate options so fluently, and why they will state a false fact with exactly the same confidence as a true one. Get this mental model right and every later decision - which tool, how hard to check, when to refuse - follows from it.
Plausible != correct. Divergent -> reach. Convergent + high stakes -> verify or do it yourself.
Why AI is good at what it is good at
Under the hood, today's generative models are pattern machines. An LLM predicts likely next words; an image model denoises toward a likely picture. They have absorbed staggering volumes of human text and images, so their sense of 'what usually comes next' is extraordinary. That is the whole source of their strengths - and it maps precisely onto tasks that are about fluency and volume rather than truth and judgement.
So AI is genuinely excellent at:
+ SPEED seconds, not hours
+ VOLUME fifty options while you think
+ DRAFTING a first version you react to and fix
+ SUMMARISING compressing long documents to their gist
+ PATTERNS spotting structure in messy data
+ AUTOMATION tireless repetition without boredomNotice what these share: in every case a fast, imperfect, plentiful output is useful, because you are going to shape it afterwards. A first draft you rewrite is worth more than a blank page. Fifty concept images you cull to three beat the four you would have sketched. A summary you spot-check saves an hour of reading. The value is real precisely where being 'roughly right and instantly available' beats being 'perfect and slow.' This is the productive half of the mental model, and it is why refusing to use AI at all is its own kind of mistake.
It is worth being concrete about how large these strengths are, because it is easy to under-rate them out of caution. An LLM can compress a hundred-page report into a page you can act on in the time it takes to fetch coffee. An image model can put forty material-palette variations in front of you before you would have finished sketching one. A summary of client feedback that would take an afternoon to collate emerges in a minute, ready for you to correct. None of this is truth on tap - but all of it is a serious multiplier on the parts of design that are about generating, drafting and digesting rather than deciding. Used on the right tasks, that multiplier is transformative, and dismissing it as mere hype is as mistaken as believing the machine can design.
Pattern machine = brilliant at fluency + volume. That is a real, usable superpower - just not truth.
Why AI is bad at what it is bad at
The same pattern-matching that powers the strengths creates the weaknesses - they are two faces of one mechanism, not a list of bugs to be patched away. A model optimised for plausibility has no separate faculty for truth, no stake in the outcome, and no access to your actual situation. So it is unreliable at:
Truth and accuracy. Models hallucinate: they generate confident, fluent, entirely false statements - a fabricated code clause, a citation to a paper that does not exist, a product with invented specs. Crucially the false output looks exactly like the true output. There is no tell in the text itself. This is the single most important limit to internalise.
Judgement of good versus merely plausible. AI can produce a competent design or argument; it cannot reliably tell you whether it is good - appropriate to this client, this site, this budget, this brief. That discrimination is the core of design and it stays yours.
Real context. The model was not at the site visit, did not feel the afternoon glare, does not know the client's marriage is strained or that the local authority is unusually strict. It knows the average of the internet, not your particular case.
Accountability. When a spec is wrong on site, a person carries the professional and legal responsibility. A model cannot. It has nothing to lose, so it cannot be trusted the way a responsible colleague can.
Genuine novelty. Because it interpolates from what exists, AI trends toward the average and the already-seen. It can recombine surprisingly, but true originality - the leap that defines memorable design - is not its strong suit. It is also prone to bias inherited from its training data, which quietly pushes output toward the conventional.
The deepest point about all five is that they are not going to be patched out by a bigger model, because they are properties of the approach, not defects in a particular product. A system whose entire objective is to produce plausible continuations has no internal sense of whether a continuation is true, good, appropriate to your case, or genuinely new - those judgements require a stake in the outcome and a grasp of the real situation that the model does not have and cannot acquire from more data. Newer models will be more fluent and often more accurate, which paradoxically makes their remaining errors harder to spot, because the surrounding prose is so convincing. So the safe professional stance is not suspicion of AI, but a permanent, calm awareness of exactly which faculties it lacks - and a workflow arranged so that a human supplies them.
The mental model: reach, or don't
Put the two halves together and you get a simple instrument for the decision you actually face: should I reach for AI on this task, and if so, how hard must I check it? Two questions settle it.
Is the task divergent or convergent? Divergent tasks open up possibilities (ideation, mood, options, first drafts); convergent tasks close down to one correct answer (a code check, a dimension, a spec, a figure). AI is a natural fit for divergent work - volume and speed are exactly what you want - and a hazard on convergent work, where a plausible-wrong answer is worse than none.
What are the stakes if it is wrong? A dud moodboard costs a click. A wrong fire-egress width costs lives and your licence. The higher the stakes, the more the burden of proof shifts onto you.
Cross those two and you get four zones: divergent and low-stakes (reach freely, curate hard, check lightly); convergent but low-stakes (let AI draft, skim before use); divergent but high-stakes (wide input, tight human selection); and convergent and high-stakes (use AI if it helps, but verify every claim to source - or just do it yourself). The tool is the same in all four; only your level of trust changes. Getting fluent at reading which zone a task sits in - often in a second, without thinking - is what separates confident AI-assisted designers from anxious ones.
The two mistakes this model guards against are opposite and equally common. One is the over-truster, who lets a confident answer settle a convergent, high-stakes question - the person who pastes an AI-summarised code clause straight onto a drawing and discovers on site that it was invented. The other is the blanket-refuser, who has heard AI 'makes things up' and therefore will not use it even to brainstorm or draft, forfeiting a genuine advantage on exactly the divergent work where a wrong answer costs nothing. Both have failed to read the zone. The model is not 'trust AI' or 'distrust AI'; it is 'trust AI this much on this task', with the dial set by divergence and stakes rather than by mood or hype. Once that becomes automatic, most of the day-to-day fear around AI simply evaporates, because you always know how much weight the output is allowed to carry.
Using strengths against limits in practice
The real art is designing your workflow so AI's strengths carry the load and its limits never get the last word. A few durable moves:
Use AI to draft, keep the truth-check human. Let it write the first pass of a spec, a proposal, a code summary - then you verify the facts against the actual source. You get the speed of a draft without inheriting its errors. A useful prompt habit is to make checking easier:
Summarise the attached fire-safety chapter for a small office. For every
requirement, quote the exact clause number and the line it comes from, so
I can verify each against the source. If a detail is not in the text, say
"not stated" rather than guessing.That last instruction does not stop hallucination, but it makes fabrication easier to catch and gives you the anchors to check.
Use volume for divergence, judgement for convergence. Generate widely when you are opening a problem up; switch to slow, careful human decision the moment you are closing it down.
Treat authoritative-looking output most sceptically. A confident number, a clean citation, a precise-sounding clause - these feel trustworthy and are exactly where hallucination hides. The polish is not evidence.
Ask what only you can bring. Before accepting any output, ask: does this reflect the real client, site and brief - the context the model never had? That question alone catches most of the failures that matter, and it keeps you the author. Modules 9.1 and 9.2 turn these instincts into rigorous evaluation and hallucination-catching routines.
Draft with AI, verify by hand. Polish is not proof. Ask: what could only I know here?
Hallucination
Confident, fluent, false output that looks identical to true output
The single most important limit; it cannot be spotted from the text alone. Module 9.2.
Divergent vs convergent tasks
Opening up possibilities vs closing to one correct answer
The first question in the reach-for-AI model; AI fits divergent, endangers convergent.
Pattern machine
Models predict plausible continuations, not verified facts
The one mechanism behind both the strengths and the limits.
Cite-to-source prompting
Asking for clause numbers / quotes so claims are checkable
Does not stop hallucination, but makes it far easier to catch. Module 1.2, 9.2.
Algorithmic bias
Training-data skew nudging output toward the conventional
Part of why AI trends to the average; watch for it in people and style. Module 9.2.
Workshop — catch a hallucination on purpose
Nothing builds the mental model faster than watching AI be confidently wrong once, on a topic you can check. This short exercise makes the abstract limit concrete and teaches the cite-to-source habit you will use for the rest of your career.
One free LLM (ChatGPT, Claude or Gemini) and access to a real source you can check against (a code excerpt, a product page, or your own knowledge).
Goal: experience hallucination first-hand and practise verifying Inputs: any free LLM; a topic you can fact-check (a code, a product, a place you know) Time: ~20 minutes
- 1Ask an LLM a specific factual question in your field where you know or can look up the true answer - e.g. a minimum staircase width in a code you have, or the specs and price of a real product.
- 2Ask it to be precise: request the exact clause number, source, and figures. Note how confident and well-formatted the answer looks.
- 3Now verify every specific claim against the real source. Mark each as correct, wrong, or invented. Look especially hard at the parts that felt most authoritative.
- 4Re-ask the same question with a cite-to-source prompt (quote the clause and line; say 'not stated' if unknown). Compare how much easier the answer is to check.
- 5Write one sentence on what this means for how you will use AI on high-stakes tasks from now on.
You’ll walk away with
A short note recording a real answer from an LLM, your verification of each claim (correct/wrong/invented), and a one-line personal rule for trusting AI on factual, high-stakes work.
Three altitudes on the same idea
Read the band that fits you — or all three.
The high-stakes, convergent end of your work is exactly where AI is weakest - so guard it hardest. Code and compliance checks, structural figures, specifications and anything going to site must be verified to source, never trusted because they read well. Meanwhile lean on AI freely at the divergent end: precedent gathering, option studies, first-draft narratives and fee letters. The judgement to tell those two ends apart, task by task, is the professional skill; the tool is the same in both.
Most of your daily work sits in AI's sweet spot - which is a gift, with one catch. Mood, palette, restyling, proposal-writing and product research are divergent or low-stakes, so generate and iterate freely. The catch is specification accuracy: dimensions, fire ratings on materials, product availability and prices are convergent facts AI will happily invent. Draft with it, then confirm every hard number against the manufacturer. Speed on the soft work; rigour on the specs.
Understanding _why_ AI fails matters more for you than any tool trick - it is what keeps you learning to design rather than leaning on a draft. Use AI to explore, summarise and get unstuck, but never to hand in an answer you cannot defend or a fact you have not checked. Practise catching hallucinations deliberately: ask for citations and verify them. That habit protects your grades now and your licence later, and it is exactly the judgement employers are testing for.
“AI keeps getting better, so hallucination and these limits are basically solved - or will be next version.”
Do it yourself
Reason it through - and be honest about the limits.
- 1In one sentence, why is AI good at drafting but bad at truth? (Hint: what is it actually optimising for?)
- 2What is a hallucination, and why can you not spot one from the text alone?
- 3Name a divergent task and a convergent task from your work, and say which one AI suits.
- 4Why should authoritative-looking output (a number, a citation) get more scepticism, not less?
- 5What can only you bring to a task that no model, however good, can?
The one line to carry out
Peer-reviewed journals & authoritative standards
- 01Hallucination (artificial intelligence) — Wikipedia, 2026.
- 02Large language model — Wikipedia, 2026.
- 03Generative artificial intelligence — Wikipedia, 2026.
- 04Algorithmic bias — Wikipedia, 2026.
We now know what AI is good and bad at, and when to reach for it. The remaining question is how to actually run the collaboration so its strengths help and its limits never win. That is the human-in-the-loop - the operating discipline for the whole course.
The author
Amogh N P
Architect, interior designer, and creative polymath. Studio Matrx began in his notebooks — his vision of design made honest, useful, and open to everyone. Its Academy is written and taught in his memory, and free, forever.
More about Amogh →