Photorealistic Material Editing Through Direct Image Manipulation

Information

• Publication Type: Journal Paper with Conference Talk
• Workgroup(s)/Project(s):
  • Path-Space Manifolds for Noise-Free Light Transport
• Date: June 2020
• Journal: Computer Graphics Forum
• Volume: 39
• Open Access: yes
• Number: 4
• Location: London, UK
• Lecturer: Karoly Zsolnai-Fehér
• ISSN: 1467-8659
• Event: Eurographics Symposium on Rendering 2020
• DOI: 10.1111/cgf.14057
• Call for Papers: Call for Paper
• Pages: 14
• Conference date: 29. June 2020 – 3. July 2020
• Pages: 107 – 120
• Keywords: neural rendering, neural networks, photorealistic rendering, photorealistic material editing

Abstract

Creating photorealistic materials for light transport algorithms requires carefully fine-tuning a set of material properties to achieve a desired artistic effect. This is typically a lengthy process that involves a trained artist with specialized knowledge. In this work, we present a technique that aims to empower novice and intermediate-level users to synthesize high-quality photorealistic materials by only requiring basic image processing knowledge. In the proposed workflow, the user starts with an input image and applies a few intuitive transforms (e.g., colorization, image inpainting) within a 2D image editor of their choice, and in the next step, our technique produces a photorealistic result that approximates this target image. Our method combines the advantages of a neural network-augmented optimizer and an encoder neural network to produce high-quality output results within 30 seconds. We also demonstrate that it is resilient against poorly-edited target images and propose a simple extension to predict image sequences with a strict time budget of 1-2 seconds per image.

Video: https://www.youtube.com/watch?v=8eNHEaxsj18

Additional Files and Images

Weblinks

• Source code
• Supplementary video
• Entry in reposiTUm (TU Wien Publication Database)
• DOI: 10.1111/cgf.14057

BibTeX

@article{zsolnaifeher-2020-pme,
  title =      "Photorealistic Material Editing Through Direct Image
               Manipulation",
  author =     "Karoly Zsolnai-Feh\'{e}r and Peter Wonka and Michael Wimmer",
  year =       "2020",
  abstract =   "Creating photorealistic materials for light transport
               algorithms requires carefully fine-tuning a set of material
               properties to achieve a desired artistic effect. This is
               typically a lengthy process that involves a trained artist
               with specialized knowledge. In this work, we present a
               technique that aims to empower novice and intermediate-level
               users to synthesize high-quality photorealistic materials by
               only requiring basic image processing knowledge. In the
               proposed workflow, the user starts with an input image and
               applies a few intuitive transforms (e.g., colorization,
               image inpainting) within a 2D image editor of their choice,
               and in the next step, our technique produces a
               photorealistic result that approximates this target image.
               Our method combines the advantages of a neural
               network-augmented optimizer and an encoder neural network to
               produce high-quality output results within 30 seconds. We
               also demonstrate that it is resilient against poorly-edited
               target images and propose a simple extension to predict
               image sequences with a strict time budget of 1-2 seconds per
               image.  Video: https://www.youtube.com/watch?v=8eNHEaxsj18",
  month =      jun,
  journal =    "Computer Graphics Forum",
  volume =     "39",
  number =     "4",
  issn =       "1467-8659",
  doi =        "10.1111/cgf.14057",
  pages =      "14",
  pages =      "107--120",
  keywords =   "neural rendering, neural networks, photorealistic rendering,
               photorealistic material editing",
  URL =        "https://www.cg.tuwien.ac.at/research/publications/2020/zsolnaifeher-2020-pme/",
}