Lesson 4.3Lesson 4.3 · Neural Capture: NeRF & Gaussian Splatting
3D Gaussian Splatting
The technique that overtook NeRF almost overnight — representing a scene as millions of tiny coloured 3D blobs that can be rendered in real time, with the same photorealism and the same honest measurement caveats
Take a scene and scatter millions of tiny, soft, coloured 3D blobs through it, each tuned until together they look exactly like the real thing — then render them fast enough to fly through live. That is Gaussian splatting.
NeRF lit the fuse, but it was slow: storing a scene inside a neural network meant querying that network millions of times to render a single image, which made real-time, fly-through-it-now experiences hard. In 2023 a different idea changed the landscape almost overnight. Instead of hiding the scene inside a network, 3D Gaussian splatting puts it out in the open as millions of explicit little blobs — and renders them fast enough to run live, at interactive frame rates, on ordinary hardware.
The effect on the field was dramatic. Within months, Gaussian splatting had largely displaced NeRF for interactive visualisation, because it delivered comparable photorealism while being fast to train and, crucially, real-time to view. For architects and designers this is the version of neural capture you are most likely to meet in a product today. This lesson builds the intuition for what a "splat" actually is, why it is so fast, and — because nothing about the neural family escapes this — why it carries exactly the same measurement cautions as NeRF.
Splatting: a scene of millions of fuzzy coloured blobs, splatted to screen in real time. Faster than NeRF, same caveat: picture, not survey.
The core intuition: a scene made of little coloured blobs
Picture the scene you want to capture, and now imagine filling it with millions of tiny, soft, semi-transparent coloured blobs — like a three-dimensional cloud of fuzzy confetti — arranged so that, seen all together, they reproduce the real scene exactly. That is a 3D Gaussian splatting model. Each blob is a 3D Gaussian: a mathematical description of a soft, fading ellipsoid. "Gaussian" just means it is densest at its centre and fades smoothly to nothing at its edges, like a soft airbrushed dot; "3D" means it is an ellipsoid sitting somewhere in space. The scene is the sum of all of them.
Each Gaussian carries a handful of properties: a position (where its centre sits in 3D space), a shape (how big it is and how it is stretched and oriented — a squashed blob can lie flat along a wall, a long one along an edge), a colour (including view-dependent colour, so it can glint differently from different angles, just as a NeRF does), and an opacity (how transparent or solid it is). Millions of these, each tuned, together paint the whole scene — broad flat blobs smoothing across walls, fine detailed ones clustering on ornament, translucent ones suggesting foliage or glass.
The crucial contrast with NeRF is explicit versus implicit. A NeRF hides the scene inside a network's weights, so you cannot point at any stored thing — you can only query the function. A Gaussian splatting model is explicit: the blobs are actual listed primitives with positions and properties, more like a supercharged, fuzzy, coloured point cloud than a neural network. You can, in principle, point at a blob, move it, delete it, or add one. This explicitness is why splats are comparatively easy to edit and manipulate, and it is half the reason they render so fast. But do not let the word "explicit" mislead you into trusting the geometry: the blobs are placed to reproduce the photographs, not to sit precisely on measured surfaces, so their collective appearance is far more reliable than the exact position of any individual blob.
Millions of soft coloured ellipsoid blobs, each with position, shape, colour, opacity. Sum them and the scene appears. Explicit, editable, fuzzy.
Why it is fast: splatting straight to the screen
The speed is the headline, so it is worth understanding where it comes from. Recall NeRF's bottleneck: to colour one pixel it casts a ray and queries a neural network at many sample points along that ray, then repeats for every pixel — an enormous number of network evaluations per frame, which is why classic NeRF rendering was slow. Gaussian splatting turns the process inside out and avoids the network-per-pixel cost entirely.
Instead of marching rays into the scene asking "what is here?", splatting takes each 3D blob and projects it forward onto the screen — it works out where that ellipsoid lands in your current view and paints its soft, fading footprint there. ("Splat" is exactly the right word: imagine throwing a soft coloured snowball at the screen and it splats into a fuzzy oval.) You sort the blobs by depth so nearer ones correctly cover farther ones, and blend their overlapping footprints by opacity. Because this is essentially rasterisation — the same fast, highly optimised operation that graphics cards (GPUs) do to draw millions of triangles in games — it runs extremely quickly, fast enough to render fresh photorealistic frames as you fly the camera around in real time.
Training a splatting model is also comparatively efficient and uses the same render-compare-adjust loop you met with NeRF: start from a rough point cloud (typically from a structure-from-motion step), render the current set of blobs, compare to the input photos, and adjust every blob's position, shape, colour and opacity to reduce the error — while also cloning and splitting blobs where more detail is needed and pruning ones that contribute nothing. The result converges faster than a classic NeRF and yields a model you can then explore instantly. So the two advantages reinforce each other: faster to build, and real-time to view. This combination — photorealism *plus* interactivity — is what made Gaussian splatting spread so rapidly and why it is the neural-capture technique most likely to appear in a tool you actually use.
Its rapid rise — and why it spread so fast
It is worth pausing on the speed of adoption, because it tells you something about the field you are entering. NeRF appeared in 2020 and drove three years of intense research; then 3D Gaussian splatting was published in 2023 and, within a matter of months, had become the default for real-time, interactive neural capture across a wave of apps, plugins and products. That is an astonishingly fast turnover for a core technology, and it is a pattern you should expect to continue: neural capture is one of the most active areas in all of computer vision right now, and the state of the art genuinely shifts on a timescale of months.
Why did splatting spread so fast in particular? Because it removed the single biggest practical barrier — speed — without giving up the thing people wanted — photorealism. A NeRF that takes hours to train and cannot be flown through in real time is a research marvel; a splat you can capture from phone video, train comparatively quickly, and then explore live, at smooth frame rates, in a browser or a headset, is a *product*. It slotted straight into the workflows architects and designers care about: immersive walkthroughs, VR and AR scenes, game-engine and web viewers, virtual tours. Its explicit, editable nature also made it friendlier to integrate with existing 3D tools than NeRF's opaque network.
The practical lesson for you is twofold. First, expect the tools to keep changing, and anchor yourself to principles — what a representation is, what it optimises for, what it can and cannot be trusted for — rather than to any one product, so you can evaluate the next breakthrough on its merits. Second, do not let the excitement erode the discipline. The very reasons splatting spread — it is fast, cheap, photoreal and easy to share — also mean it is easy to over-trust. A client or a colleague flying smoothly through a gorgeous real-time splat of a building will naturally assume it is accurate. Your job is to know, and to say, that it is a visualisation of superb fidelity and uncertain metric truth — which brings us to the caveat that applies to splatting exactly as it did to NeRF.
The same metric caveat: fast and beautiful is still not measured
Everything this module has said about appearance versus measurement applies to Gaussian splatting without exception, and it is essential to say so clearly precisely because splatting is so convincing and so interactive that it invites misplaced trust. A splat is optimised to reproduce the input photographs, just like a NeRF. The blobs are positioned so that their combined, blended appearance matches the photos from the captured angles — not so that any blob sits exactly on a real surface at a true coordinate. The representation is explicit, but that does not make it *metric*.
The consequences are the familiar ones. Scale is not inherent: a raw splat from phone video does not know the real size of the room unless scale is imposed from outside. Geometry is approximate: the exact 3D position of the surface implied by a cloud of overlapping fuzzy blobs is fuzzy too — a wall that looks crisp in the render may not sit on a clean plane, thin objects and edges are uncertain, and regions the cameras saw poorly are filled with blobs that look plausible rather than measured. Artefacts appear: stray or "floating" blobs, smeared regions, and view-dependent tricks that collapse when you move somewhere the cameras never went. And as with NeRF, there is no honest accuracy figure attached — only a compelling impression of realism, made more compelling still by the real-time interactivity.
So the rule is unchanged and non-negotiable, and it is the same rule that runs through this entire course: a Gaussian splat is a visualisation, not a survey. Use it — genuinely, enthusiastically — for presentation, immersive walkthroughs, VR/AR, design-review context, and capturing visual richness. Do not dimension from it for coordination, setting-out, fabrication, boundaries or structural decisions. Verify any load-bearing number against laser scanning or controlled photogrammetry, and hand anything legally or structurally binding to a licensed surveyor working to recognised standards and verified equipment specifications. Research on extracting accurate meshes from splats, imposing and checking scale, and tying them to control is advancing quickly — so treat this caveat as a fast-moving snapshot and re-verify current capabilities, but until a specific tool demonstrably meets the accuracy a job requires, proven against independent measured data, keep splatting for the picture and measured capture for the measurement.
Real-time and gorgeous raises the temptation to trust it. Don't. Splats are tuned to match photos, not surfaces. Scale external, geometry fuzzy. Verify.
Explicit but not metric
What the blobs represent
Gaussians are tuned to reproduce photos, not to sit on measured surfaces; explicit primitives are not survey coordinates. Principles here; measured models come from laser/photogrammetry.
No inherent scale
Real-world units
A raw splat has no metric scale unless imposed externally; scaling and georeferencing stay survey tasks, deferred to control and a licensed surveyor (Module 1.3-1.4).
Over-trust risk
Real-time interactive realism
Smooth, photoreal fly-throughs invite misplaced confidence; state clearly it is a visualisation. Verify any decision-driving dimension against measured capture (Module 9).
Fast-moving field
Currency of capabilities
Gaussian splatting (2023) displaced NeRF for real-time use within months; mesh extraction and scaling research advance monthly. Re-verify current claims rather than trusting this snapshot.
Workshop — compare a splat and a measured capture of the same space
The fastest way to feel the appearance-versus-measurement gap is to look at both for one real space. This workshop structures that comparison; if you cannot capture, you run it as a reasoned thought experiment.
A notebook and a space to study. Optional but valuable: a phone with a free Gaussian-splatting app plus any measured reference (tape measure, phone LiDAR, or one known dimension) to quantify the gap.
Goal: experience where a splat delights and where it must not be trusted Inputs: a room or object + this lesson + a notebook; optional: a phone with a free Gaussian-splat app, and any measured reference (tape, phone LiDAR, or a known dimension) Time: ~50 minutes
- 1Capture or imagine a Gaussian splat of a space: if capturing, shoot steady video covering all surfaces from many positions; note how long capture, training and first interactive view take, and contrast that with what a NeRF or a laser scan would take.
- 2Explore it and record the delights: list what the splat conveys beautifully — materials, reflections, light, atmosphere, fine detail — and the real-time experience of flying through it.
- 3Hunt for the failures: look for floating blobs, smeared or fuzzy surfaces, edges that dissolve, and regions that break up when you move to viewpoints the camera never took; note each and link it to how splats are built.
- 4Test a measurement: pick two dimensions (e.g. a door width and a ceiling height), read them off the splat, then obtain them by a measured method (tape, phone LiDAR, or a known reference) and compare; record the discrepancy and whether it would matter for fit-out or coordination.
- 5Write the verdict: one paragraph on what you would confidently use this splat for, what you would never use it for, and where a licensed surveyor would be required instead.
You’ll walk away with
A one-page side-by-side: the splat's visual strengths, its observed artefacts, a measured-versus-splat comparison for two dimensions with the discrepancy noted, and a clear statement of acceptable and unacceptable uses. Keep it as direct evidence for the 4.4 placement argument.
Three altitudes on the same idea
Read the band that fits you — or all three.
Gaussian splatting is the neural capture you are most likely to deploy — a real-time, photoreal, fly-through model of a site or building, captured from video and shown in a browser or headset. It is outstanding for immersive client and jury presentation, design-review context and VR/AR, and its explicit, editable nature integrates more readily with 3D tools than NeRF. But it is governed by the same rule: the blobs are tuned to reproduce photos, not to sit on measured surfaces, so scale is external and geometry is approximate. Keep it as the experiential layer over a metric model built from laser scanning or controlled photogrammetry; never dimension from the splat for coordination, setting-out or fabrication; and defer anything binding or structural to a licensed surveyor. Watch the field — capabilities change monthly.
For interiors, a Gaussian splat is the most shareable, immersive record you can make of a space — walk a client through a real-time photoreal model of their room from a phone video. Materials, light, reflections and atmosphere come through vividly, and the speed means no long waits for a result. Use it freely for presentation, before/after storytelling and virtual tours. Just hold the line on measurement: the splat will not reliably give you true ceiling heights, alcove widths or out-of-square angles, because its geometry is fuzzy and unscaled. Capture the dimensions that joinery and fit-out depend on with measured methods (phone LiDAR with checks, a tape, commissioned photogrammetry), and treat the splat as the beautiful walkthrough, not the tape measure. Flag binding items for survey.
Gaussian splatting is the hottest technique in neural capture right now, and explaining why it is fast is a sharp way to show you understand the field. Hold the picture: the scene is millions of explicit, soft, coloured 3D blobs (position, shape, colour, opacity) that are projected and rasterised straight to the screen — no per-pixel network query — which is why it renders in real time, unlike ray-marched NeRF. Understand that it is trained the same way (match the photos) and therefore carries the same limits: external scale, approximate geometry, artefacts, no honest error figure. Be able to compare it with NeRF on representation, speed, editing and the shared metric caveat. You are not expected to implement it; you are expected to reason clearly about what it is and when to trust it — which is for the picture, not the measurement.
“Gaussian splatting stores the scene as explicit 3D primitives with real positions, and it runs in real time like a proper 3D model — so unlike a vague NeRF, a splat is basically an accurate, measurable point cloud you can dimension from.”
Do it yourself
No tools needed — reason from the lesson.
- 1Describe a 3D Gaussian and the properties it carries (position, shape, colour, opacity), and how millions of them form a scene.
- 2Explain why splatting renders in real time when classic NeRF does not — what does it do instead of querying a network per pixel?
- 3Contrast NeRF and Gaussian splatting on representation (implicit vs explicit), speed, and editability.
- 4Why does being explicit and real-time NOT make a splat metrically accurate?
- 5A client is flying through a gorgeous real-time splat of their building and asks you for the width of a room. What do you say and do?
The one line to carry out
Peer-reviewed journals & authoritative standards
- 01Gaussian splatting — Wikipedia — Gaussian splatting, 2026.
- 02Neural radiance field — Wikipedia — Neural radiance field, 2026.
- 03Novel view synthesis — Wikipedia — Novel view synthesis, 2026.
- 04Computer vision — Wikipedia — Computer vision, 2026.
- 05Deep learning — Wikipedia — Deep learning, 2026.
We now have both neural techniques in hand — NeRF and Gaussian splatting — and a consistent verdict on their limits. The final lesson of the module draws the honest conclusion: exactly where neural capture belongs in real practice, where it does not yet belong, and how it complements rather than replaces the measured capture that the rest of this course is built on.
The author
Amogh N P
Architect, interior designer, and creative polymath. Studio Matrx began in his notebooks — his vision of design made honest, useful, and open to everyone. Its Academy is written and taught in his memory, and free, forever.
More about Amogh →