Lesson 0.1Lesson 0.1 · Foundations — How Generative AI Sees Design
How Generative AI Sees Design
What the model is really doing when it makes an image
The model that just drew your building has never seen a building.
When Midjourney returns a gorgeous concept in thirty seconds, it feels like the machine understood you. It did not. A generative image model has no concept of a wall, a load path, a room, or a client. What it has is a statistical memory of how pixels tend to arrange themselves in millions of captioned pictures - and a process for pulling a new, plausible arrangement out of pure noise. Grasp that one idea and everything downstream - why prompts work, why it hallucinates, why you must stay the designer - falls into place.
Appearance without logic. Say it until it's instinct - it's the whole course in four words.
It starts with static, and subtracts
The dominant image models today - Stable Diffusion, Midjourney, Flux, DALL·E - are diffusion models, and the name is the whole idea. In training, the model is shown a real image and watches it get destroyed step by step into random static (that's the diffusion). It learns to run that tape backwards: given a noisy image and a text description, predict what the slightly-less-noisy version looked like.
At generation time it starts from pure noise - a screen of random static seeded by a number - and denoises it, a little at a time, nudged at every step toward the meaning of your prompt, until a coherent picture emerges. Nothing is 'drawn'. An image is sculpted out of randomness by repeatedly asking 'what would make this look more like the words modern courtyard house, warm evening light?'
Not a painter filling a canvas - a sculptor removing noise until an image is left.
It knows pixels and captions, not architecture
The model learned from a colossal set of image-plus-text pairs scraped from the web. From that it built an intuition for correlations: the words 'brutalist', 'jaali', 'travertine', 'golden hour' each pull the image toward regions of a vast internal map where pictures with those captions tend to live.
But it never learned why. It does not know that a cantilever needs a counterweight, that a stair riser has a comfortable height, that a kitchen wants a work triangle, or that your site faces west. It knows that images labelled 'cantilever' look a certain way. This is the single most important thing to internalise: the model reproduces the appearance of good architecture without any of its logic. That is exactly why it is brilliant for ideation and dangerous for documentation.
Same words, different picture — and why that's a feature
Ask the same prompt twice and you get two different images. That's not a bug; it's the noise. Each generation begins from a different random seed, and because the model is probabilistic - it samples a plausible answer rather than computing the one correct answer - the seed steers which of countless valid images you land on.
For a designer this is a superpower and a discipline. A superpower because one prompt becomes a generator of variations - you fish an idea out of a hundred. A discipline because reproducibility takes effort: fix the seed and you can hold a composition steady while you change one word, which is how you move from slot-machine luck to deliberate control. Modules 1 and 3 are entirely about converting that randomness into intent.
The seed is the difference between gambling and steering. Learn to hold it.
Why the mental model changes how you work
If you believe the AI 'understands' design, you'll ask it to be the architect - and it will confidently hand you a beautiful, unbuildable lie: a stair to nowhere, a structure that can't stand, a plan that doesn't close. If you understand it denoises appearance from patterns, you'll use it correctly: to explore, to communicate, and to accelerate the front of the process, while every decision about function, structure, code and context stays with you.
The rest of this course is built on that stance. You will learn to prompt precisely, to control the output with your own drawings, to direct materials and light, and to fold AI into a real workflow - always as an instrument the designer plays, never as the designer.
Stable Diffusion (Latent Diffusion Model)
Open-source text-to-image diffusion model
The open foundation of much of the ecosystem; runs locally, fully controllable, the basis for ControlNet and LoRAs later in the course.
Midjourney
Hosted text-to-image, strong out-of-the-box aesthetics
Closed and subscription-based, but the fastest route to beautiful architectural imagery; used throughout Module 2.
Diffusion process (forward + reverse)
The core mechanism of modern image models
Training destroys images into noise; generation reverses it. Understanding this explains prompts, seeds and hallucination.
Seed
The random starting point of a generation
Fix it to reproduce or steer an image; vary it to explore. Your first lever of control.
Workshop — watch the model think
You don't need to install anything to feel how this works. Using any text-to-image tool you can access (Midjourney, a free Stable Diffusion space, Adobe Firefly, or Studio Matrx's own DesignAI), run a tiny experiment that makes diffusion, seeds and pattern-matching visible in your own hands.
Any text-to-image tool (Midjourney, a free Stable Diffusion web space, Adobe Firefly, or Studio Matrx DesignAI). No installation or payment required to complete the exercise.
Goal: prove to yourself that the model works from patterns and randomness, not understanding Inputs: one text-to-image tool + one prompt you'll reuse Time: ~25 minutes
- 1Write one clear prompt for a building or room you can picture, e.g.
a small modern courtyard house, warm evening light, architectural photograph. Generate it four times without changing a word. - 2Look at the four results side by side. They're all 'right' and all different - that's the seed and the probabilistic sampling. Note what stayed constant (mood, style) versus what drifted (layout, details).
- 3Now add a term the model will recognise as a pattern - a style or material, e.g. add
brutalist concreteorKerala vernacular, sloping tiled roof. Watch the whole image shift toward that region of what it has seen. - 4Deliberately provoke a hallucination: ask for something structurally specific, like
a 40-metre cantilever with no visible support, section drawing. Observe how confidently it produces something that could never be built - appearance without logic. - 5If your tool exposes the seed, lock it and regenerate the same prompt: the image should return nearly identical. Change one word and watch only that aspect move. This is control in miniature.
You’ll walk away with
A small board of your own generations - the four variations, the style shift, and the confident hallucination - annotated with one line each on what it taught you about how the model 'sees'.
Three altitudes on the same idea
Read the band that fits you — or all three.
Treat generative AI as a concept-stage instrument, not a design authority. It compresses days of mood exploration into an afternoon and communicates intent to clients fast - but it has no structural or code logic. Use it to widen the option space early; keep buildability, program and context firmly in your own hands and your BIM/CAD tools.
This is where AI is most immediately useful to you. Because interiors are so much about appearance - palette, material, light, styling - a model that reproduces appearance is a natural fit for moodboards, restyling and client options. Understanding it works from patterns tells you why it nails a 'Japandi living room' vibe but invents impossible joinery you'll need to correct.
Understanding _how_ these tools work is what separates a skilled practitioner from someone pasting prompts. Employers can tell the difference. Grasp diffusion, latent space and seeds now and every later lesson - prompting, ControlNet, workflow - will make sense as technique rather than magic, and your portfolio will show judgement, not just pretty pictures.
“The AI understood my brief and designed the building.”
Do it yourself
No tool needed - reason it through.
- 1In one sentence, explain what a diffusion model does at generation time.
- 2Why do you get a different image each time you run the same prompt?
- 3Give one example of something the model reproduces the appearance of but has no understanding of.
- 4What is a seed, and why would a designer want to fix it?
- 5Complete the rule: use generative AI for _ and _; never for _.
The one line to carry out
Peer-reviewed journals & authoritative standards
- 01Ho, J., Jain, A., & Abbeel, P. — Denoising Diffusion Probabilistic Models — Advances in Neural Information Processing Systems (NeurIPS), 2020.
- 02Rombach, R., Blattmann, A., Lorenz, D., Esser, P., & Ommer, B. — High-Resolution Image Synthesis with Latent Diffusion Models — IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2022.
- 03Goodfellow, I., et al. — Generative Adversarial Networks — Advances in Neural Information Processing Systems (NeurIPS), 2014.
If the model responds to the _patterns_ in your words, then the words are your controls - so next we take prompting apart: the anatomy of a design prompt and the exact vocabulary that moves the image.
The author
Amogh N P
Architect, interior designer, and creative polymath. Studio Matrx began in his notebooks — his vision of design made honest, useful, and open to everyone. Its Academy is written and taught in his memory, and free, forever.
More about Amogh →