G-Style: Stylized Gaussian Splatting

Information

• Publication Type: Journal Paper (without talk)
• Workgroup(s)/Project(s): not specified
• Date: 2024
• Article Number: e15259
• DOI: 10.1111/cgf.15259
• ISSN: 1467-8659
• Journal: Computer Graphics Forum
• Number: 7
• Pages: 13
• Volume: 43
• Publisher: WILEY
• Keywords: Artificial intelligence, Computer graphics, Neural networks

Abstract

We introduce G-Style, a novel algorithm designed to transfer the style of an image onto a 3D scene represented using Gaussian Splatting. Gaussian Splatting is a powerful 3D representation for novel view synthesis, as—compared to other approaches based on Neural Radiance Fields—it provides fast scene renderings and user control over the scene. Recent pre-prints have demonstrated that the style of Gaussian Splatting scenes can be modified using an image exemplar. However, since the scene geometry remains fixed during the stylization process, current solutions fall short of producing satisfactory results. Our algorithm aims to address these limitations by following a three-step process: In a pre-processing step, we remove undesirable Gaussians with large projection areas or highly elongated shapes. Subsequently, we combine several losses carefully designed to preserve different scales of the style in the image, while maintaining as much as possible the integrity of the original scene content. During the stylization process and following the original design of Gaussian Splatting, we split Gaussians where additional detail is necessary within our scene by tracking the gradient of the stylized color. Our experiments demonstrate that G-Style generates high-quality stylizations within just a few minutes, outperforming existing methods both qualitatively and quantitatively.

Additional Files and Images

Weblinks

• Entry in reposiTUm (TU Wien Publication Database)
• DOI: 10.1111/cgf.15259

BibTeX

@article{kovacs-2024-gsg,
  title =      "G-Style: Stylized Gaussian Splatting",
  author =     "Áron Samuel Kov\'{a}cs and Pedro Hermosilla and Renata
               Raidou",
  year =       "2024",
  abstract =   "We introduce G-Style, a novel algorithm designed to transfer
               the style of an image onto a 3D scene represented using
               Gaussian Splatting. Gaussian Splatting is a powerful 3D
               representation for novel view synthesis, as—compared to
               other approaches based on Neural Radiance Fields—it
               provides fast scene renderings and user control over the
               scene. Recent pre-prints have demonstrated that the style of
               Gaussian Splatting scenes can be modified using an image
               exemplar. However, since the scene geometry remains fixed
               during the stylization process, current solutions fall short
               of producing satisfactory results. Our algorithm aims to
               address these limitations by following a three-step process:
               In a pre-processing step, we remove undesirable Gaussians
               with large projection areas or highly elongated shapes.
               Subsequently, we combine several losses carefully designed
               to preserve different scales of the style in the image,
               while maintaining as much as possible the integrity of the
               original scene content. During the stylization process and
               following the original design of Gaussian Splatting, we
               split Gaussians where additional detail is necessary within
               our scene by tracking the gradient of the stylized color.
               Our experiments demonstrate that G-Style generates
               high-quality stylizations within just a few minutes,
               outperforming existing methods both qualitatively and
               quantitatively.",
  articleno =  "e15259",
  doi =        "10.1111/cgf.15259",
  issn =       "1467-8659",
  journal =    "Computer Graphics Forum",
  number =     "7",
  pages =      "13",
  volume =     "43",
  publisher =  "WILEY",
  keywords =   "Artificial intelligence, Computer graphics, Neural networks",
  URL =        "https://www.cg.tuwien.ac.at/research/publications/2024/kovacs-2024-gsg/",
}