AI View Synthesis for Glasses-Free 3D: NeRF, 3D Gaussian Splatting, and the Multi-View Pipeline

Glasses-free 3D displays split into two content pipelines. Eye-tracked two-view displays serve one viewer with a left-eye view and a right-eye view — any source of a stereoscopic frame pair drives them, and content cost is low. Multi-view and light field displays generate eight, sixteen, forty-five, or sixty simultaneous views and fan them across a viewing cone — content cost is high.

The bottleneck for the multi-view path has never been the optics. It is the content pipeline. Traditional workflows render one 3D scene from 45 to 60 camera angles per frame. GPU cost scales linearly with view count, and authoring complexity scales with it.

AI view synthesis attacks exactly this bottleneck. It takes sparse input — a few photographs, a monocular video, or even a single image — and synthesizes the missing views with a neural or learned representation. The output drops into a multi-view display’s quilt format with a fraction of the traditional render cost.

This page covers the four techniques that matter for glasses-free 3D content in 2026, their real performance numbers, their deployment paths into light field displays like the Looking Glass family, and where they still fall short. For the optical side of light field displays, see the light field display page.

Why Multi-View Content Was Historically Hard

A 4K panel running 45 light field views requires 45 perspective renders per frame. For a static product visualization with simple geometry, this is manageable. For organic subjects, dynamic motion, or anything sourced from existing 2D photographs and video, it is a wall.

The traditional alternatives each have costs:

Pre-rendered quilts. Render the 45 views once, store the quilt, play it back. Works for static content, breaks for interactive or dynamic content.
Real-time multi-view rendering. Render all 45 views from a real-time 3D scene every frame. Requires a powerful GPU and a renderer with native multi-view output.
Multi-camera rigs. Capture 45 simultaneous video streams from 45 cameras. Hardware-intensive, calibration-sensitive, hard to deploy for archival or in-the-wild content.

AI view synthesis is the fourth option: synthesize the missing views from sparse input using a learned scene representation.

The Four Techniques That Matter

NeRF (Neural Radiance Fields, 2020)

How it works. A multi-layer perceptron maps a 5D input — a 3D spatial coordinate plus a 2D viewing direction — to a color and a volume density. Volume rendering accumulates these values along each camera ray to produce a pixel. The scene is represented entirely by the weights of the MLP.

Reference. Mildenhall, Srinivasan, Tancik, Barron, Ramamoorthi, Ng — Representing Scenes as Neural Radiance Fields for View Synthesis, ECCV 2020.

Performance. On an NVIDIA V100, the original NeRF trains a single scene in one to two days and renders at roughly one frame per thirty seconds at low resolution. Instant-NGP (Müller et al., SIGGRAPH 2022) brought training time down to seconds using hash-grid encodings, but real-time inference on consumer GPUs remained out of reach.

What it meant for glasses-free 3D. NeRF proved that a small set of photographs could synthesize arbitrary novel views at high visual quality. It did not deliver the real-time inference that interactive multi-view applications need.

3D Gaussian Splatting (3DGS, 2023)

How it works. A scene is represented as a large set of explicit 3D Gaussian primitives — each defined by a position, a covariance matrix, an opacity, and spherical-harmonic coefficients that describe view-dependent appearance. The primitives are projected to image space with a tile-based differentiable rasterizer and blended with front-to-back alpha compositing. There is no neural network inside the representation. Optimization updates the Gaussian parameters directly from photometric loss against training views.

Reference. Kerbl, Kopanas, Leimkühler, Drettakis — 3D Gaussian Splatting for Real-Time Radiance Field Rendering, SIGGRAPH 2023.

Performance. Training on a single GPU takes minutes to tens of minutes for a typical scene. Real-time rendering at 1080p at 30+ fps is achievable on consumer GPUs (RTX 3070 and above) once trained. This is the technique that made NeRF-quality view synthesis practical for interactive applications.

What it meant for glasses-free 3D. 3DGS is the first view-synthesis technique that delivers both training tractability and real-time inference on consumer hardware. It is the technique that made light field content production move from a research curiosity to a deployable workflow.

4D Gaussian Splatting (4DGS, 2024)

How it works. Extends 3D Gaussian Splatting to dynamic scenes by adding a time dimension to the Gaussian primitives. The representation can capture motion and deformation across time, enabling view synthesis from monocular or sparse multi-view video input.

Reference. Multiple papers in 2024 — Wu et al., Yang et al., and others — proposed 4D Gaussian Splatting variants for dynamic scene capture.

Performance. Training time scales with scene length and complexity. Real-time inference is achievable on consumer GPUs for short sequences.

What it meant for glasses-free 3D. 4DGS opens the door to capturing live subjects, performances, and dynamic scenes for light field display — content categories that traditional multi-camera rigs handled poorly and that static 3DGS could not represent.

Monocular Depth Estimation + DIBR

How it works. A monocular depth estimation model predicts a depth map from a single image or video frame. Depth-image-based rendering (DIBR) then warps the source image to produce novel views by reprojecting pixels according to their predicted depth.

Reference. Multiple architectures — MiDaS, Depth Anything, ZoeDepth, Metric3D — for monocular depth. DIBR is a long-standing computer graphics technique.

Performance. Real-time inference on a modern GPU. Quality depends heavily on the depth estimator — current 2025 models produce depth maps suitable for moderate-quality novel view synthesis.

What it meant for glasses-free 3D. DIBR with monocular depth is the cheapest path to multi-view content. It does not deliver the photorealism of 3DGS, but it runs at real-time frame rates on commodity hardware and works with arbitrary 2D sources — photographs, archival video, even live streams.

Performance Summary

Technique	Training	Inference Quality	Real-Time on Consumer GPU	Best Fit
NeRF	Hours	High	No (without acceleration)	Static, archival
Instant-NGP	Seconds	High	Marginal	Static, prototyping
3DGS	Minutes	High	Yes	Static and slow-changing scenes
4DGS	Tens of minutes	High	Yes	Dynamic scenes
DIBR + Monocular Depth	None	Moderate	Yes	Live 2D-to-3D conversion

These are approximate ranges as of mid-2026. Performance continues to improve as new architectures ship.

Deployment Paths into Light Field Displays

Quilt-Based Workflow

A light field display consumes a quilt — a single high-resolution image containing N view tiles in a specific layout. For the Looking Glass family, N is typically 45 views, and the quilt layout is documented in the Looking Glass Bridge SDK.

The end-to-end pipeline looks like:

Capture or render a scene. Either a real-world subject captured with photos / video, or a synthetic 3D scene rendered from a real-time engine.
Train a view-synthesis model. A NeRF, 3DGS, or 4DGS model is optimized against the captured views.
Render novel views. At inference time, the model synthesizes the 45 (or N) views needed for the quilt.
Compose the quilt. A small post-processing step interleaves the views into the quilt layout.
Output to the display. Looking Glass Bridge or a plugin (Unity, Unreal, Blender) sends the quilt to the display.

Real-Time Playback

For interactive applications, the entire pipeline must run at frame rate. 3DGS and 4DGS hit this threshold on consumer GPUs for most scenes. DIBR with monocular depth hits it easily but at lower visual quality.

Static Playback

For archival content, product visualizations, or museum installations, the quilt can be pre-rendered and stored. The display plays back the quilt like a video. This is the cheapest deployment path.

Real Product Impact

Looking Glass + 3DGS

Recent research has demonstrated 3DGS rendering directly onto Looking Glass hardware at 45 views. The visual quality approaches traditional multi-view rendering while cutting the GPU cost substantially. A team that previously needed a high-end workstation to render 45 views of a complex scene can now do it on a single consumer GPU.

Looking Glass + NeRF

Earlier work showed NeRF-based rendering onto Looking Glass at lower view counts. The visual quality is high but the inference cost is the bottleneck. For static or slowly-changing content, this is workable.

Eye-Tracked Displays + AI 2D-to-3D Conversion

Eye-tracked displays do not strictly require AI view synthesis because they only need two views (left and right). However, AI 2D-to-3D conversion — using monocular depth estimation to generate a stereo pair from a single image or video — extends the value of existing 2D content libraries. Samsung’s Odyssey 3D and Acer SpatialLabs products use AI-based 2D-to-3D conversion for this reason.

Content for Eyetracked Displays

For eye-tracked displays, the cheaper and more common path is to produce SBS stereo content directly from a real-time 3D scene. Most DICOM viewers, NDT inspection suites, CAD packages, and game engines output SBS natively. AI view synthesis is the better fit when the source content is 2D — photographs, archival video, or live streams that need to become 3D.

Remaining Limitations

Transparent and reflective surfaces. NeRF and 3DGS struggle with view-dependent effects like glass and mirrors. Production content with significant glass and reflection requires careful scene capture or falls back to traditional rendering.
Dynamic motion at high frame rates. 4DGS handles slow and moderate motion well. Rapid motion with complex deformation still requires careful capture and may not reach interactive frame rates on consumer GPUs.
Calibration sensitivity. Multi-view training from a sparse camera rig requires accurate camera calibration. Errors propagate into the synthesized views. Monocular capture is more robust but produces lower-quality output.
Color and lighting consistency. AI view synthesis can drift in color or lighting across synthesized views, producing visible seams. Production pipelines include color-matching post-processing to suppress this.
Storage and bandwidth. A trained 3DGS model is typically hundreds of megabytes to a few gigabytes. Streaming trained models at scale adds bandwidth and storage considerations.

What This Means for a Buyer

If you are evaluating a light field display for a deployment in 2026:

Content pipeline matters more than optics. A Looking Glass with no content pipeline is a beautiful exhibit with nothing to show. A Looking Glass with a working 3DGS or NeRF pipeline is a deployable system.
Real-time content needs consumer-GPU-class hardware. An RTX 3070 or better handles most 3DGS workloads at interactive frame rates. Lower-end GPUs can play back pre-rendered quilts.
Static content is cheaper. Pre-rendered quilts work for museum installations, product showcases, and archival content where interactivity is not required.
2D content can be converted. DIBR + monocular depth is the cheapest path from existing 2D libraries to multi-view content. Quality is moderate but the workflow is real-time.
Eye-tracked displays benefit less directly. AI view synthesis matters less for eye-tracked displays because they only need two views. The SBS stereo path is simpler. AI 2D-to-3D conversion is the relevant technique for eye-tracked displays that need to use existing 2D content.

Where to Go Next

For the optical side of light field displays: Light field display technology.
For how light field compares to eye-tracked: Light field vs eye-tracked comparison.
For the broader technology landscape: Glasses-free 3D display overview.
For product-level detail on light field products: Looking Glass product page.
For workflow applications: Content creation use case.