Lesson 3.4Lesson 3.4 · Control — Making AI Obey
img2img, Inpainting & Outpainting
Refining, fixing and extending - the finishing controls
The first render is a draft. These three controls are how you finish it.
A conditioned render gets you a strong first pass - but design lives in the second, third and tenth. The window's wrong. That corner needs a different material. The frame is too tight; you want to see more of the street. img2img, inpainting and outpainting are the finishing controls that let you refine what you have instead of re-rolling from scratch: restyle a whole image while keeping its bones, regenerate just one region, or grow the canvas beyond its edges. This is where AI stops producing pictures and starts supporting iteration.
Stop re-rolling. Mask small, dial low, edit the flaw, keep the win. That's direction.
img2img and the fidelity dial
Plain text-to-image starts from pure noise. img2img starts from your image instead: it adds some noise back onto an existing picture, then denoises toward your prompt. Because it began from your image rather than static, the output keeps that image's structure - and the single dial that governs how much survives is denoising strength, a value from 0 to 1.
At low strength (around 0.2 to 0.4) the model barely changes anything - perfect for a gentle restyle, a material swap, a lighting shift while the composition holds rock-steady. At mid strength (0.5 to 0.6) it reinterprets more freely but still respects the overall layout - the sweet spot for turning a rough conditioned pass into a polished render. Push it high (0.75 and up) and it keeps only a loose echo of the original before drifting toward an essentially new scene. Strength is your fidelity dial: turn it down to stay faithful, up to reinvent. A worked feel for the numbers: at 0.3 a grey concrete room becomes the same room warmed to travertine with the furniture untouched; at 0.55 the palette, textiles and light all reinterpret while the sofa stays where it sits; at 0.85 you have gambled the room away for a new one that merely rhymes with it. The reliable habit is to start low and creep up - it is far easier to add change than to recover a composition you have already dissolved. Master this one dial and you can specify, in advance, exactly how much of the original you are willing to spend.
Low strength = a nudge. High strength = a new idea wearing your image's clothes.
Inpainting - regenerate exactly one region
Often you love the render except for one thing - a garbled window, an awkward tree, a material you want changed on a single wall. Inpainting is the surgical fix: you paint a mask over just that region, and the model regenerates only inside the mask while leaving every pixel outside it untouched. Prompt what the region should become, and the model fills it to blend seamlessly with its surroundings, reading the light and style at the mask's edge so the patch belongs.
This is the control that makes AI renders usable in practice. Instead of re-rolling a whole image and losing everything good about it to fix one flaw, you edit locally - exactly like retouching, but generative. Inpainting is how you correct hallucinations (a stair that leads nowhere, a mullion that can't exist), swap elements (this sofa for that one, this door for a jaali screen), or refine a detail without disturbing the rest. A few settings decide whether the patch reads or betrays itself. Mask feathering softens the mask edge so new pixels graduate into the old instead of showing a seam. Masked denoising strength works just like img2img: keep it low (about 0.4) for a subtle change, higher for a full replacement. And 'inpaint only masked' (at full resolution) tells the tool to work at high detail inside the mask rather than smearing a whole-image pass across it. Combine inpainting with ControlNet and you can even hold the geometry inside the masked region - restyle a window's glazing while a Canny or MLSD map keeps its exact frame.
Outpainting - grow beyond the frame
The mirror image of inpainting extends outward. Outpainting masks new, empty canvas around your existing image and asks the model to invent what continues past the original edge - more of the street, more sky above the parapet, the room the doorway leads into. The original stays as an anchor; the model reads its style, light and perspective and grows a plausible extension that lines up with what is already there.
For designers this solves two everyday problems. First, reframing: a render cropped too tight becomes a wider establishing shot without re-rendering the building - you simply grow the sky and foreground it needed. Second, context: a building generated in isolation gets a setting - neighbours, landscape, a foreground path - stitched believably onto what you already have, so it reads as sited rather than floating. The technique rewards patience. Extend a strip at a time - say 20 to 30% of the width per pass - so each new region has plenty of the original beside it to match perspective and light against; ask for a huge extension in one go and the far edge drifts into invention that no longer agrees with your building. Keep the overlap generous, prompt the continuation (quiet residential street, mature trees, early evening), and feather the seam. Like inpainting, outpainting is just masked denoising - the same trick pointed at the space beyond the picture rather than a hole inside it. Two patient passes usually turn a claustrophobic crop into a composed, sited image.
Inpaint fills a hole; outpaint grows the edge. Same trick, opposite direction.
The refinement loop - how the pieces work together
None of these is a one-shot button; together they're a loop, and that loop is the real design workflow. A typical pass: condition a render from your drawing (Lesson 3.3); img2img at low-to-mid strength to polish the whole thing and unify its light; inpaint to fix the two or three regions that came out wrong; outpaint to reframe or add context; then round again, feeding the improved image back in as the new starting point. Each pass keeps what works and edits only what doesn't - the exact opposite of the slot-machine re-roll you spent Lesson 3.1 fighting.
The discipline is restraint and judgement. Use the lowest denoising strength that does the job; mask tightly so you disturb only what must change; and after every step, check the result against your intent, not just its prettiness. Version as you go, too - save each pass as a numbered file, because a bold edit will sometimes go wrong, and a non-destructive history is what lets you step back a move and try another, exactly like layers and undo in any mature design tool. A concrete run might read: condition (geometry locked), img2img 0.35 (whole image polished), inpaint the ground-floor window (strength 0.5, ControlNet holding the frame), inpaint the foreground planting, outpaint the sky and street in two strips, then img2img 0.25 for a final unifying pass. Six deliberate moves, each logged, each reversible.
This is the whole promise of Module 3 delivered: from a tool that gave you a different building every time, to an instrument you can steer, correct and extend with the precision a real project demands. Conditioning set the bones; img2img polished the whole; inpainting fixed the flaws; outpainting grew the frame. String them together and you have a repeatable pipeline that carries design intent from first sketch to final image. You are no longer prompting and hoping - you are directing.
Where the loop goes wrong - and how to save it
Three problems separate a clean finish from an obvious edit. The inpainted patch looks pasted on. Almost always the mask hugged the object too tightly, giving the model no surrounding context to blend against, or the masked strength ran too high. Feather the edge, include a little of the surroundings in the mask, drop the strength, and make sure the inpaint prompt matches the scene's light and material - 'warm evening light' inside a mask you filled under 'overcast' will never sit right. img2img erased something you wanted. The strength was too high for the job; step it down and run again from the previous version, not the ruined one - which is exactly why you kept numbered saves. The outpaint drifted into a different building. You extended too far in one pass; undo, take a narrower strip, and keep more overlap so the new region has your geometry to agree with.
Two rules keep the whole loop honest. First, change one variable at a time - strength, or mask, or prompt, never all three - so you always know which move helped. Second, treat every generative edit as a proposal to be checked, not a result to be trusted: overlay against your intent and reject the beautiful edit that quietly moved a real thing. Done this way the loop is genuinely non-destructive and genuinely directed - you keep the wins, discard the misses, and arrive at a finished image you can defend line by line, because you know exactly what each pass did and why.
Patch looks pasted? Feather the mask, lower the strength, match the light. Then re-check.
img2img
Generating from an existing image instead of pure noise
Keeps the source's structure; how much is set by denoising strength - your fidelity dial.
Denoising strength
How much noise is added back before re-denoising
Low (0.2-0.4) barely changes the image; mid (0.5-0.6) restyles while holding layout; high (0.8+) drifts to a new scene.
Inpainting
Regenerating only a masked region of an image
Surgical local edits - fix hallucinations, swap elements, refine details - while the rest stays untouched. Pairs with ControlNet.
Outpainting
Extending the canvas beyond the original edges
Reframe a tight crop or add context; works best a strip at a time so each new region matches the original.
Workshop - the full refinement loop
Take a render (ideally your conditioned one from Lesson 3.3) all the way to finished using every control in this lesson. Use Stable Diffusion with an inpainting model (Automatic1111 or ComfyUI), or DesignAI's edit tools if you prefer the browser.
Stable Diffusion with an inpainting model (Automatic1111 or ComfyUI), or Studio Matrx DesignAI's edit tools. A render to refine - ideally your Lesson 3.3 output.
Goal: turn a rough first pass into a finished, directed image Inputs: one render you already have + an img2img/inpaint-capable tool Time: ~50 minutes
- 1Start from an existing render. Run img2img at LOW denoising strength (~0.3) to polish the whole image - note how little changes. Then try ~0.6 and see the layout hold while surfaces reinterpret. Keep the version you prefer.
- 2Find the worst region - a garbled window, an awkward object. Mask it tightly and INPAINT with a prompt for what it should be. Regenerate until it blends; keep the mask small so nothing else moves.
- 3Swap one element deliberately: mask a piece of furniture or a wall finish and inpaint a replacement. Confirm the rest of the image is pixel-identical outside the mask.
- 4OUTPAINT to reframe: mask a strip of new canvas on one side and extend the scene - more street, more sky, or the adjacent room. Do it in one or two strips so it matches.
- 5Feed the improved image back into img2img at low strength for a final unifying pass. Lay first-pass and final side by side and annotate every edit you made.
You’ll walk away with
A before/after pair - your rough first pass and the directed final - with callouts for each move: the strength you used, what you inpainted, and where you outpainted.
Three altitudes on the same idea
Read the band that fits you — or all three.
These controls turn a promising render into a submission-ready one. img2img polishes a conditioned pass without losing the scheme; inpainting fixes the hallucinated details that would otherwise disqualify an image (the stair to nowhere, the impossible mullion); outpainting reframes a tight crop or drops the building into its street context. The rule that keeps you honest: mask tightly, use the lowest strength that works, and check every edit against the design, never against how pretty it looks. Log every move - the strength, the mask, the ControlNet you held inside it - so a submission image is one you can defend line by line rather than one you got lucky on.
Inpainting is the single most useful AI control you'll own. Keep the room, mask the sofa, prompt a new one - true 'same room, different piece' options at last. Swap a wall finish, add a rug, change the pendant, all without re-rolling the space. img2img at low strength restyles a whole room's palette while holding the layout; outpainting extends a cropped shot so the client sees the full space. This is the workflow clients think AI already does - now you can actually deliver it. Master mask feathering and a low masked strength (about 0.4) and the pasted-on look disappears, which is the whole difference between an edit a client believes and one they don't.
Showing a refinement loop - condition, img2img, inpaint, outpaint - is what separates a portfolio that 'used AI' from one that COMMANDS it. Present a first pass beside the finished image and annotate each edit: what you masked, what strength you used, what you fixed. That transparency proves iterative control and design judgement. Employers can tell instantly whether you re-rolled until something looked good or actually directed the tool to a decision. Present the first pass beside the finished image with a callout on each edit; it proves you directed the tool to a decision instead of re-rolling until something happened to look good.
“If a render comes out wrong, I should just regenerate the whole thing until it's right.”
Do it yourself
No tool needed - reason it through.
- 1In one sentence, how does img2img differ from text-to-image at the start?
- 2You want to restyle a room's palette but keep its exact layout. Roughly what denoising strength, and why?
- 3What does inpainting change, and what does it leave untouched?
- 4Give one design task outpainting solves that inpainting cannot.
- 5Describe the refinement loop in the right order using all three controls.
The one line to carry out
Peer-reviewed journals & authoritative standards
- 01Rombach, R., Blattmann, A., Lorenz, D., Esser, P., & Ommer, B. - High-Resolution Image Synthesis with Latent Diffusion Models — IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2022.
- 02Zhang, L., Rao, A., & Agrawala, M. - Adding Conditional Control to Text-to-Image Diffusion Models — IEEE/CVF International Conference on Computer Vision (ICCV), 2023.
- 03Podell, D., et al. - SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesis — arXiv preprint, 2023.
- 04comfyanonymous - ComfyUI: A powerful and modular Stable Diffusion GUI — GitHub repository, 2024.
That completes Module 3 - you can now make AI obey, from conditioning your drawings to refining every region. Take the mastery check, then Module 4 puts it all to work on the core production skill: turning drawings into renders, plans to 3D, elevations to facades, massing to concepts.
The author
Amogh N P
Architect, interior designer, and creative polymath. Studio Matrx began in his notebooks — his vision of design made honest, useful, and open to everyone. Its Academy is written and taught in his memory, and free, forever.
More about Amogh →