Neural Radiance Fields (NeRF)

Start with the name, because it is unusually descriptive. A field, in physics, is something that has a value at every point in space — the temperature in a room is a field, defined at every location. A radiance field assigns, to every point, the light leaving that point. A Neural Radiance Field uses a neural network to represent that field. Put together: a NeRF is a function that, given any point in the scene (a position x, y, z) and a direction you are looking from, returns two things — the colour seen at that point from that direction, and the density, a measure of how much solid stuff is there (empty air has near-zero density; the surface of a wall has high density).

Notice what is and is not stored. There is no explicit list of surface points, no mesh, no coordinates of the wall. Instead the entire scene lives inside the weights of a small neural network — the learned numbers that make the function give the right answers. To find out what is at a location, you do not look it up in a list; you *run the network* with that location as input and read off the colour and density it computes. Because a neural network is a smooth, continuous function, you can query it at infinitely fine resolution, anywhere, which is why NeRFs can render crisp images and handle soft, gradual things well.

This is a genuinely different mental model from everything earlier in the course. A point cloud is a bag of measurements; a mesh is a surface; both are explicit things you can open and inspect. A NeRF is implicit — the scene is encoded indirectly, as the behaviour of a function, and you only ever see it by asking it to render a view. That implicitness is the source of both its magic (compact, continuous, photorealistic) and its difficulty (you cannot simply read a wall's coordinates out of it, and its geometry can be vague). Hold this picture — a function that knows the colour and solidity of every point — and the rest of NeRF follows from it.

Given that function, how do you actually make a picture? Through volume rendering, and the mechanism is intuitive. To work out the colour of one pixel in your desired new view, you imagine a ray shooting from the camera, through that pixel, into the scene. You walk along the ray and, at many sample points, ask the network: colour and density here? Near empty air the density is tiny and contributes almost nothing; where the ray passes through a solid surface the density spikes and that point's colour dominates. Blend all the samples along the ray, weighting each by its density and by whether something solid already blocked the view in front of it, and you get the final pixel colour. Do that for every pixel and you have rendered the whole image. Because colour is returned *as a function of viewing direction*, a shiny surface can send different colours toward different cameras — which is exactly how NeRFs reproduce reflections and gloss.

So much for rendering a scene the network already knows. How does it *learn* the scene in the first place? By training, in the classic machine-learning loop. You supply the input photographs, each labelled with the camera position it was taken from (these poses usually come from a photogrammetry-style structure-from-motion step first). The network starts knowing nothing; it renders its guess of each training photo using the ray process above, compares its render to the real photograph, measures the error, and nudges its weights to reduce that error. Repeat this millions of times across all the photos and the network gradually converges on a field that reproduces every training view — and, because the field is smooth and consistent, it then generalises to render *new* views convincingly.

This training loop is the key to everything, including the cautions that follow. The network is optimised for one thing only: making its renders match the input photographs. Nothing in the process measures or rewards real-world distances. The scene representation that results is whatever best reproduces those images — which is usually visually excellent and geometrically approximate. Understanding that the objective is photo-reproduction, not measurement, explains every strength and every weakness of NeRF in the next section.

NeRF's strengths fall straight out of how it works, and they are real. First and foremost, photorealism. Because the network is trained to reproduce actual photographs and can be queried continuously, a good NeRF renders new views that look like genuine photos — soft shadows, subtle colour gradients, fine texture, atmospheric depth — far beyond the somewhat clinical look of a coloured point cloud or a textured mesh. For conveying the *feel* of a space to a client, this is transformative.

Second, view-dependent appearance, which is NeRF's signature and a thing classical methods genuinely cannot match. Because colour is a function of viewing direction, materials behave as they do in reality: a polished marble floor, a glazed tile, a metal fitting, a glass screen or a pool of water show moving highlights and reflections as you orbit the scene. Bake a single colour onto each point of a mesh and all of that is lost; a NeRF keeps it. This also lets NeRF handle soft, fuzzy and semi-transparent subjects — foliage, hair, thin grilles, carved stone screens (jali), lace — that scatter laser beams and confuse photogrammetry. The density field represents "partly there" gracefully.

Third, compactness and accessibility. The entire scene can live in a neural network whose stored weights are far smaller than a dense multi-billion-point cloud of the same space — an elegant kind of compression. And the input is just photographs or video from an ordinary camera or phone, with no specialist hardware. For a student, a designer or a practice exploring the technology, that low barrier to entry matters: you can make a genuine NeRF of a courtyard, a sculpture or a room with equipment you already own. Taken together, these strengths make NeRF outstanding for visualisation, immersive presentation, virtual and augmented reality, and the documentation of visually rich or delicate subjects — the places where *looking right* is the goal. The catch, always, is that looking right is not the same as measuring right, which is the subject of the final section.

Now the honest debit side, because a professional chooses a method knowing its failure modes. The first practical weakness is training time and compute. Classic NeRF does not produce a result the instant you stop capturing; it must train, historically for many minutes to hours on a capable GPU, because it runs that render-compare-adjust loop millions of times. Faster variants and better hardware have shortened this dramatically, and Gaussian splatting (Module 4.3) attacks exactly this problem, but NeRF's heritage is "capture now, wait for the model" rather than the real-time feedback a laser scanner or a splat gives. For fieldwork where you want to confirm coverage on the spot, that lag matters.

The second is data hunger and capture discipline. A NeRF needs many photographs, reasonably well distributed around the subject, with good overlap, consistent lighting, and accurately estimated camera poses. Too few images, big gaps in coverage, motion blur, changing light or moving objects, and the result degrades into floaters (wisps of spurious density hanging in the air), blurred regions, and gaps where the network simply had too little to learn from. Large, complex or poorly-lit scenes are hard. As with all capture, the result is only as good as the input, and NeRF is fairly demanding about that input.

The third weakness is the one that governs everything in this module: metric reliability. Because training optimises for reproducing photographs, the recovered geometry is approximate and the scene has no inherent real-world scale — a raw NeRF does not know if the column is two metres or twenty metres tall unless scale is imposed from outside (a known dimension, survey control, or the structure-from-motion step's own limitations). Surfaces that render perfectly can be subtly non-planar in the model; thin objects smear; unseen regions are plausibly invented. Extracting a clean, correct mesh or point cloud from a NeRF is an active research problem, not a solved one. So the rule stands, unchanged from 4.1: a NeRF is a superb visualisation and an untrustworthy ruler. Use it to *see*; verify any dimension that matters against laser scanning or controlled photogrammetry; and send anything binding to a licensed surveyor. These limits are also a snapshot — NeRF research moves fast — so re-check current capabilities rather than treating today's weaknesses as permanent.