Researchers at Simon Fraser University (SFU) have developed a method that allows users to manipulate lighting in a photograph with the same precision found in 3D graphics tools like Blender or Unreal Engine. By moving away from 'black box' generative AI models that guess lighting based on prompts, this system enables creators to place and adjust specific light types—such as spot lights, point lights, or environmental illumination—directly within a scene.

This image illustrates the core capability of the system to manipulate lighting while maintaining physical accuracy.
Moving Beyond Generative AI 'Black Boxes'
Current generative AI tools often struggle with relighting because they rely on large-scale neural networks to predict how light might interact with a scene. This often results in unpredictable outputs where the AI may stray from the original content of the photograph, forcing users into a cycle of prompt engineering and trial-and-error.
The SFU approach replaces this guesswork with explicit physical control. By simulating the physics of light, the researchers ensure that the relighting result is a physically accurate rendition of the user's specific creative vision.
“By simulating the lights, we ensure our result is a physically accurate rendition of the user's desired lighting.”
— CS Researchers Develop A New Tool That Brings Blender-like Lighting Control to Any Photograph
The Reconstruction and Rendering Pipeline
The process begins by inferring a colored mesh representation of a scene from a single photograph. This 3D model captures the shape and surface colors of the objects while intentionally stripping away the original lighting information.

This visual supports the technical explanation of the reconstruction phase where 3D models are established.
Once the 3D scene is established, users can place virtual light sources into the environment. A path-tracing engine then simulates these lights to create a rough preview of the scene. Finally, a feed-forward neural renderer transforms that rough preview into a photorealistic photograph, blending physical accuracy with high-fidelity appearance.
Training and Future Applications
To ensure the system works on diverse, real-world images, the team developed a self-supervised training methodology. This uses differentiable rendering to reconstruct scene illumination, allowing the neural renderer to be trained on raw image collections without needing manually labeled data.
While the tool is currently limited to static images, the researchers intend to expand it to video. This could significantly reduce costs for independent filmmakers and content creators by allowing them to adjust lighting after a scene is captured, eliminating the need for expensive lighting gear or multiple reshoots.