Studio Matrx Monthly · Volume 1 · Issue 4 · September 2026
Amogh N P
 In loving memory of Amogh N P — Architect · Designer · Visionary 
The Neural Turn in CaptureLesson 4.1
Reality Capture & Scan-to-BIM/Module 4 · Neural Capture: NeRF & Gaussian Splatting

Lesson 4.1 · Neural Capture: NeRF & Gaussian Splatting

The Neural Turn in Capture

For a decade capture meant measuring geometry directly; now machine learning reconstructs whole scenes and invents new viewpoints from a handful of photographs, and the results are breathtaking, useful and quietly treacherous for measurement

12 min Interactive lessonFree · open lessonByAmogh N P· Architect & interior designer
The hook

A machine that has seen thirty photos of a room can now show you that room from a viewpoint no camera ever occupied — and make it look utterly real. That is the neural turn, and it changes what capture can mean.

For the first three modules of this course, capture meant one honest idea: to know where a surface is, you measure the distance to it, directly, with a laser pulse or with the geometry of overlapping photographs. The output was explicit — a point cloud, a mesh, coordinates you could dimension. You could trust a number because a number had been measured.

The neural turn breaks that assumption in the most interesting way. Instead of measuring geometry and storing it, you hand a machine-learning model a set of ordinary photographs and let it *learn the scene* — learn how light leaves every surface in every direction well enough to paint the scene convincingly from viewpoints no photo ever captured. The results, from Neural Radiance Fields (NeRF) in 2020 to 3D Gaussian splatting in 2023, are so photorealistic they can be mistaken for video. They are genuinely useful. And they carry a caveat you must hold from the first sentence: looking real and being metrically true are not the same thing, and this family is strong on the former while unreliable on the latter.

The neural turn: photos in, a learned scene out, new views invented. Beautiful to look at. Verify before you measure.

What actually changed: from measuring to learning

To see what is new, hold the old methods clearly in mind. Laser scanning measures distance directly: fire a pulse, time its return, record a point. Photogrammetry measures indirectly but still geometrically: find the same feature in many overlapping photos and triangulate where it must be in space. Both end in the same currency — explicit 3D geometry, points and surfaces with coordinates you can measure against. The computer stores *where things are*.

Neural capture stores something different: it stores *how the scene looks*. You feed a machine-learning model many photographs of a subject, each tagged with the camera position it was taken from, and the model trains until it can reproduce those photos. In doing so it learns an internal representation of the scene's colour and structure rich enough to render the scene from a brand-new camera position it was never shown. This is novel view synthesis — inventing a plausible photograph from a viewpoint that never existed. The two landmark techniques are NeRF (2020), which stores the scene inside the weights of a small neural network, and Gaussian splatting (2023), which stores it as millions of little coloured 3D blobs; Modules 4.2 and 4.3 take each in turn.

The shift is profound. Classical capture answers "where is this surface?" with a measured coordinate. Neural capture answers "what would a camera see from here?" with a rendered image. The first is a ruler; the second is a painter with an extraordinary memory. Both are built from the same raw ingredient — photographs or scans of the real world — and both belong to the broad field of computer vision and 3D reconstruction. But their outputs are not interchangeable. A point cloud is a measurement you can audit; a neural render is an appearance you must take on trust until you verify it. Understanding that one-sentence difference is the whole foundation of this module, and the reason a beautiful neural capture is never, by itself, a survey.

Two roads from photos to 3D EXPLICIT GEOMETRY vs LEARNED SCENE CLASSICAL - builds geometry you can measure Photos or laser scans SfM / MVS or range measurement Point cloud / mesh explicit coordinates Take dimensions NEURAL - learns a scene, renders new views Many photos Train a neural scene model Render novel views photoreal images Looks real measure w/ care Classical output is measurable coordinates. Neural output is a renderer - stunning images, but metric geometry and real-world scale are not guaranteed without control and verification.
Zoom
Two roads from photographs to 3D. The classical pipeline ends in explicit, measurable geometry; the neural pipeline ends in a renderer that produces photorealistic new views but whose metric geometry and scale are not guaranteed.

Old capture stores WHERE things are. Neural capture stores HOW the scene LOOKS. One is a ruler; the other is a painter with perfect memory.

Why it is exciting: photorealism and new viewpoints

The excitement is real and worth naming precisely, because neural capture is not hype for its own sake — it does things the older methods genuinely struggle with. The headline is photorealism. A laser scan of a carved temple column gives you exact geometry but a rather clinical, sometimes patchy appearance; the colour is sampled coarsely and the subtle way light plays across the stone is lost. A neural capture of the same column can look like a film of it — every highlight, every soft shadow, the sheen of polished surfaces, all reproduced so faithfully that a viewer forgets they are looking at a reconstruction.

The second gift is view-dependent appearance, the thing that makes surfaces look alive. Real materials do not look the same from every angle: a glazed tile, a mirror, a lacquered door, water, a glass facade all change as you move. Classical geometry-plus-texture handles this badly, because a single colour is baked onto each point. A NeRF learns colour *as a function of viewing direction*, so reflections and glints move correctly as you orbit the scene — something point clouds and meshes simply cannot do. It also means neural methods cope gracefully with the soft, fuzzy, semi-transparent things that wreck laser scans and photogrammetry: foliage, hair, lace screens, fine grilles, thin edges.

Third, it is accessible. The raw input is just photographs or ordinary video — no lakhs-rupee scanner, often just a phone. For visualisation, presentation, virtual and augmented reality, marketing walkthroughs, and capturing the *visual richness* of a space that a point cloud flattens, neural capture is extraordinary: you can hand a client an immersive, fly-anywhere, photoreal model of their building captured in an afternoon. That is a capability the profession did not have five years ago, and it is why the field has moved so fast. Hold the excitement — and keep reading, because the same properties that make it dazzling are exactly what make it unreliable for measurement.

Photorealism is not metric reliability PHOTOREALISM / VISUAL RICHNESS -> METRIC RELIABILITY -> Laser scanning (LiDAR) high accuracy, plainer visuals Photogrammetry measurable, good texture NeRF Gaussian splatting dazzling views, scale not guaranteed trust for dimensions trust for the picture
Zoom
Photorealism is not metric reliability. Placing methods on two independent axes shows laser scanning high on trustworthy measurement, neural methods high on visual realism, and photogrammetry bridging the two.

Why it needs care: appearance is not measurement

Here is the discipline this whole module exists to instil. The very mechanism that makes neural capture beautiful — optimising a model until it *reproduces the photographs* — optimises for appearance, not for metric truth. The model is rewarded for rendering images that match the input photos. It is not rewarded for getting the real-world distance between two walls correct. So a NeRF or a splat can look flawless from every trained angle while its underlying geometry is soft, locally distorted, or simply invented in places the cameras did not see well.

Three specific cautions follow. First, scale is not inherent. Classical capture ties into real units through measured distances or survey control; a raw neural capture from phone photos has no idea whether the room is three metres wide or thirty, unless scale is imposed externally. Second, geometry can be confidently wrong. Neural methods produce "floaters" (wisps of density hanging in empty air), smear thin objects, and guess at hidden or poorly-photographed regions in ways that look plausible but are not measured. A flat wall may render perfectly yet not be flat in the model. Third, there is no honest error figure you can read off. A survey-grade laser scan comes with a stated accuracy; a neural capture comes with a subjective impression of realism, which is not the same thing and can be dangerously persuasive.

The professional consequence is simple and non-negotiable, and it echoes the whole course: treat a neural capture as a visualisation, not a survey. If a dimension off a neural model will drive a cut, a fabrication, a boundary, a setting-out or a structural decision, you do not trust it — you verify that dimension against a laser scan, a photogrammetric model built with control, a tape, or a licensed surveyor. Neural capture is a superb way to *see* a place and a poor way to *measure* it, and the rest of this module (4.2, 4.3, 4.4) unpacks exactly why, method by method, and where it does belong.

Photorealism is not metric reliability PHOTOREALISM / VISUAL RICHNESS -> METRIC RELIABILITY -> Laser scanning (LiDAR) high accuracy, plainer visuals Photogrammetry measurable, good texture NeRF Gaussian splatting dazzling views, scale not guaranteed trust for dimensions trust for the picture
Zoom
Photorealism is not metric reliability. Placing methods on two independent axes shows laser scanning high on trustworthy measurement, neural methods high on visual realism, and photogrammetry bridging the two.

The model is trained to MATCH THE PHOTOS, not to get distances right. Looks perfect, measures unreliable. Verify every load-bearing number.

Where the neural turn sits in capture — and in this module

It helps to place neural capture on the same map as the methods you already know, rather than treating it as a rival that wins or loses outright. Think of two independent axes. One is photorealism and visual richness — how convincingly the result reproduces the look of the real place. The other is metric reliability — how much you can trust a dimension taken from it. Laser scanning sits high on metric reliability and modest on photorealism: superb for measurement, visually plain. Photogrammetry sits in the useful middle: measurable when done with control, with good texture. Neural capture sits high on photorealism and, for now, low on guaranteed metric reliability: dazzling to look at, not to be dimensioned blindly. No method is "best"; each occupies a different corner, and mature practice chooses by what the job needs — or combines them, as Module 4.4 will show.

There is one more property of this field you must internalise: it moves extraordinarily fast. NeRF is from 2020; Gaussian splatting, which largely eclipsed it for real-time use, is from 2023; the literature churns monthly. Techniques that impose real-world scale, extract clean meshes, remove floaters, and tie neural captures to survey control are active, rapidly improving research. So every specific limitation stated in this module is a *snapshot as of writing*, not a permanent verdict. The honest position is to learn the principles — why appearance and measurement diverge, what each representation is and is not — so that when the tools improve, you can judge the new claims for yourself rather than believing a vendor demo.

Across the next three lessons we go deeper: 4.2 builds an intuition for NeRF — a scene as a continuous learned field of colour and density. 4.3 does the same for Gaussian splatting — the faster, real-time representation that took the field by storm. 4.4 draws the honest conclusion: where neural capture genuinely belongs (visualisation, presentation, VR/AR, view synthesis, hard-to-scan visual richness), where it does not yet belong (anything metric or binding), and how it complements rather than replaces measured capture. Keep the course's through-line in view throughout: capture is measurement, you must always know and check the accuracy, and binding results go to a licensed surveyor.

Two roads from photos to 3D EXPLICIT GEOMETRY vs LEARNED SCENE CLASSICAL - builds geometry you can measure Photos or laser scans SfM / MVS or range measurement Point cloud / mesh explicit coordinates Take dimensions NEURAL - learns a scene, renders new views Many photos Train a neural scene model Render novel views photoreal images Looks real measure w/ care Classical output is measurable coordinates. Neural output is a renderer - stunning images, but metric geometry and real-world scale are not guaranteed without control and verification.
Zoom
Two roads from photographs to 3D. The classical pipeline ends in explicit, measurable geometry; the neural pipeline ends in a renderer that produces photorealistic new views but whose metric geometry and scale are not guaranteed.
Verify-this: enjoy the realism, but never mistake a neural render for a measurement

Appearance vs accuracy

What a neural capture optimises for

Training matches photographs, not distances; realism is not a stated accuracy. Treat output as visualisation. Principles here; binding accuracy follows verified specs and a surveyor.

Scale & georeferencing

Real-world units and position

Raw neural captures have no inherent scale or coordinate frame; imposing and checking them is a survey task. Defer georeferencing to control and a licensed surveyor (Module 1.3-1.4).

Verification of dimensions

Any number that drives a decision

Cross-check load-bearing dimensions against laser scanning / controlled photogrammetry / a tape, never off the neural view alone. Module 9 on quality and accuracy.

Fast-moving field

Currency of any accuracy claim

NeRF (2020) and Gaussian splatting (2023) evolve monthly; re-verify current capabilities and vendor claims rather than treating this snapshot as permanent.

Hands-on workshop

Workshop — sort a set of capture tasks by what each one actually needs

The core judgement of this whole module is knowing when photoreal appearance is enough and when measured geometry is mandatory. In this workshop you build that instinct by triaging realistic tasks, before you touch any neural tool.

Just a notebook and your own project experience. No software or capture hardware — this lesson builds judgement, not a pipeline.

Given & goal
Goal: reliably separate 'needs measurement' from 'needs a good picture'
Inputs: the task list below (or your own projects) + this lesson + a notebook
Time: ~35 minutes
  1. 1List 8-10 realistic capture tasks across your work: e.g. a client walkthrough of a heritage haveli, a fit-out survey for new joinery, a marketing video of a completed villa, setting-out for a steel extension, documenting a carved facade, a VR tour for a competition, checking a floor is level, an insurance record of a space.
  2. 2For each, write the single decisive question: 'what will someone DO with this output?' and decide whether it must carry trustworthy dimensions or only needs to look real.
  3. 3Tag each task 'metric' (needs measured geometry), 'visual' (needs photoreal appearance), or 'both', and note which capture method you would use for the metric part (laser / photogrammetry) and which for the visual part (NeRF / splatting).
  4. 4Pick the one task most likely to be captured neurally by mistake, and write two sentences on what could go wrong if someone dimensioned from that neural model.
  5. 5Write a one-paragraph rule for yourself: a plain-language test you will apply in future to decide whether a neural capture is acceptable for a given job, and where you would still call a licensed surveyor.

You’ll walk away with
A one-page triage table: each task tagged metric / visual / both, the method chosen for each part, the one dangerous case flagged with its failure mode, and your personal rule for when neural capture is and is not acceptable. Keep it; it is the decision framework Module 4.4 formalises.

The worked example

Three altitudes on the same idea

Read the band that fits you — or all three.

For the architectCapturing sites and buildings as the reliable basis for design

Neural capture gives you a new, cheap, photoreal way to record and present a site or building — but it does not give you measurable geometry, and confusing the two is where it bites. Use it for immersive client walkthroughs, design-review context, heritage visual documentation and VR/AR, captured from a phone or camera in hours. Do not let a dimension from a NeRF or a splat drive coordination, setting-out, fabrication or a structural decision: for the metric basis of the project you still commission laser scanning or controlled photogrammetry, and you still scan-to-BIM from that. The skilled move is to run the two together — a metric scan for the model, a neural capture for the experience — and to know, every time, which one a given number came from. Defer anything binding, georeferenced or legally consequential to a licensed surveyor.

For the interior designerAccurate existing interiors, as-builts and fit-out verification

For interiors, neural capture is a beautiful way to show a space and a risky way to measure one. A phone-shot NeRF or Gaussian splat of a room can give a client a stunning, fly-through record of an existing interior, its materials, its light, its character — richer than any point cloud for conveying atmosphere, and invaluable for mood, presentation and before/after storytelling. But when you need the real dimensions to design joinery, fit-out or furniture that actually fits — ceiling heights, out-of-square walls, the true opening for a wardrobe — do not trust numbers pulled off the neural view. Verify against a measured capture (even a careful tape or phone-LiDAR scan with checks) and flag anything binding for professional survey. Capture the look neurally; capture the measurements properly.

For the studentHow the real world becomes measured 3D data and models

This is the newest, fastest-moving frontier of reality capture, and understanding it clearly — especially its limits — marks you out. Learn the one idea that matters: classical capture measures and stores geometry, while neural capture learns to reproduce appearance and synthesise new views, which makes it photorealistic but not reliably metric. Be able to explain why training a model to match photographs optimises for looking right, not measuring right, and why scale, floaters and invented geometry are the result. You are not expected to publish NeRF research; you are expected to reason about when a photoreal capture is trustworthy for dimensions (rarely, not yet) versus for visualisation (superbly), and to know that a beautiful render is never, by itself, a survey.

Misconception check

NeRF and Gaussian splatting are just a better, newer kind of 3D scanning — they produce photorealistic models from cheap photos, so they make laser scanning and photogrammetry obsolete and you can measure off them exactly as you would a point cloud.

They are not a replacement for measured capture; they solve a different problem. Laser scanning and photogrammetry measure and store explicit geometry — coordinates you can audit and dimension, with a stated accuracy. Neural methods are trained to reproduce the photographs they were given, which optimises for appearance, not for metric truth. The consequences are concrete: the model has no inherent real-world scale unless one is imposed externally; its geometry can be locally distorted, smeared on thin objects, or invented in unseen regions while still looking flawless; it generates floaters and artefacts; and it ships with no honest error figure, only a persuasive impression of realism. A NeRF or splat can therefore look utterly convincing from every angle while being metrically unreliable. That does not make neural capture useless — it is superb for photoreal visualisation, novel-view synthesis, VR/AR, presentation and capturing visual richness (reflections, foliage, fine detail) that laser scans handle poorly. It makes it the wrong tool for measurement. Treat it as a visualisation, verify any load-bearing dimension against a laser scan or controlled photogrammetry, and hand anything binding — boundaries, setting-out, structural or deformation work, georeferenced deliverables — to a licensed surveyor. The field is also moving fast, so re-check specific accuracy claims rather than assuming today's limits are permanent.
Try it

Do it yourself

No tools needed — reason it through from the lesson.

  1. 1In one sentence each, contrast what laser scanning, photogrammetry and neural capture store about a scene.
  2. 2What is novel view synthesis, and why is it something classical point clouds cannot do?
  3. 3Explain why training a model to reproduce photographs optimises for appearance rather than for metric accuracy.
  4. 4Name three specific reasons a photorealistic neural capture can be metrically unreliable (e.g. scale, floaters, invented geometry).
  5. 5Give one task where a NeRF or splat is the right tool and one where it would be a dangerous mistake, and say why.
Take this with you

The one line to carry out

The neural turn lets machine learning reconstruct scenes and synthesise new, photorealistic viewpoints from ordinary photos — dazzling for visualisation, presentation and VR/AR, but optimised for appearance rather than measurement, so its geometry and scale are not reliably metric; enjoy the realism, verify every load-bearing dimension against measured capture, and hand anything binding to a licensed surveyor.
Take it further
References & further reading

Peer-reviewed journals & authoritative standards

  1. 013D reconstructionWikipedia — 3D reconstruction, 2026.
  2. 02Novel view synthesisWikipedia — Novel view synthesis, 2026.
  3. 03Computer visionWikipedia — Computer vision, 2026.
  4. 04Machine learningWikipedia — Machine learning, 2026.
  5. 05PhotogrammetryWikipedia — Photogrammetry, 2026.
Related lessons
Recap
Neural capture is a genuinely new family in reality capture: instead of measuring and storing explicit geometry the way laser scanning and photogrammetry do, it uses machine learning to learn a scene's appearance from many photographs and then synthesise convincing new viewpoints the camera never occupied. The two landmark techniques are NeRF (2020), which stores the scene in a neural network's weights, and 3D Gaussian splatting (2023), which stores it as millions of coloured blobs. It is exciting because it is photorealistic, reproduces view-dependent effects like reflections and gloss, handles soft and fuzzy detail that defeats scanners, and can be captured cheaply from a phone. It needs care because the same training objective that makes it beautiful — reproducing the input photos — optimises for looking right, not measuring right: scale is not inherent, geometry can be distorted or invented while still rendering flawlessly, artefacts like floaters appear, and there is no honest error figure. On a map of photorealism versus metric reliability, neural capture sits high on the first and, for now, low on the second. The professional rule is to treat it as a visualisation, verify any load-bearing dimension against measured capture, and defer binding results to a licensed surveyor — while remembering this is a fast-moving field whose specific limits are a snapshot, not a permanent verdict.
Carry forward →

We have framed the neural turn and its promise and peril in the abstract. Next we open the first technique and build a real intuition for how it works: a Neural Radiance Field that stores an entire scene as a continuous, learned field of colour and density.

A

The author

Amogh N P

Architect, interior designer, and creative polymath. Studio Matrx began in his notebooks — his vision of design made honest, useful, and open to everyone. Its Academy is written and taught in his memory, and free, forever.

More about Amogh →