Information
- Publication Type: Journal Paper with Conference Talk
- Workgroup(s)/Project(s):
- Date: July 2019
- Journal: Computer Graphics Forum
- Volume: 38
- Open Access: yes
- Number: 4
- Lecturer: Adam Celarek
- Event: Eurographics Symposium on Rendering 2019
- DOI: 10.1111/cgf.13775
- Call for Papers: Call for Paper
- Publisher: The Eurographics Association and John Wiley & Sons Ltd.
- Conference date: 10. July 2019 – 12. July 2019
- Pages: 111 – 121
- Keywords: measuring error, light transport, global illumination
Abstract
This paper proposes a new methodology for measuring the error of unbiased physically based rendering algorithms. The current state of the art includes mean squared error (MSE) based metrics and visual comparisons of equal-time renderings of competing algorithms. Neither is satisfying as MSE does not describe behavior and can exhibit significant variance, and visual comparisons are inherently subjective. Our contribution is two-fold: First, we propose to compute many short renderings instead of a single long run and use the short renderings to estimate MSE expectation and variance as well as per-pixel standard deviation. An algorithm that achieves good results in most runs, but with occasional outliers is essentially unreliable, which we wish to quantify numerically. We use per-pixel standard deviation to identify problematic lighting effects of rendering algorithms. The second contribution is the error spectrum ensemble (ESE), a tool for measuring the distribution of error over frequencies. The ESE serves two purposes: It reveals correlation between pixels and can be used to detect outliers, which offset the amount of error substantially.Additional Files and Images
Additional images and videos
Additional files
paper_preprint:
Paper
presentation:
Slides from the presentation at EGSR
supplemental_material:
Additional examples, experiments and similar.
Weblinks
- Git repository
Git repository with the implementation of the error spectrum ensemble and short rendering generator. - Entry in the publication database of TU-Wien
- DOI: 10.1111/cgf.13775
BibTeX
@article{celarek_adam-2019-qelta, title = "Quantifying the Error of Light Transport Algorithms", author = "Adam Celarek and Wenzel Jakob and Michael Wimmer and Jaakko Lehtinen", year = "2019", abstract = "This paper proposes a new methodology for measuring the error of unbiased physically based rendering algorithms. The current state of the art includes mean squared error (MSE) based metrics and visual comparisons of equal-time renderings of competing algorithms. Neither is satisfying as MSE does not describe behavior and can exhibit significant variance, and visual comparisons are inherently subjective. Our contribution is two-fold: First, we propose to compute many short renderings instead of a single long run and use the short renderings to estimate MSE expectation and variance as well as per-pixel standard deviation. An algorithm that achieves good results in most runs, but with occasional outliers is essentially unreliable, which we wish to quantify numerically. We use per-pixel standard deviation to identify problematic lighting effects of rendering algorithms. The second contribution is the error spectrum ensemble (ESE), a tool for measuring the distribution of error over frequencies. The ESE serves two purposes: It reveals correlation between pixels and can be used to detect outliers, which offset the amount of error substantially.", month = jul, journal = "Computer Graphics Forum", volume = "38", number = "4", doi = "10.1111/cgf.13775", publisher = "The Eurographics Association and John Wiley & Sons Ltd.", pages = "111--121", keywords = "measuring error, light transport, global illumination", URL = "https://www.cg.tuwien.ac.at/research/publications/2019/celarek_adam-2019-qelta/", }