Reducing the Memory Footprint of 3D Gaussian Splatting

Information

• Publication Type: Journal Paper (without talk)
• Workgroup(s)/Project(s):
  • Instant Visualization and Interaction for Large Point Clouds
• Date: May 2024
• Article Number: 16
• DOI: 10.1145/3651282
• ISSN: 2577-6193
• Journal: Proceedings of the ACM on Computer Graphics and Interactive Techniques
• Number: 1
• Pages: 17
• Volume: 7
• Publisher: Association for Computing Machinery (ACM)
• Pages: 1 – 17
• Keywords: 3D gaussian splatting, memory reduction, novel view synthesis, radiance fields

Abstract

3D Gaussian splatting provides excellent visual quality for novel view synthesis, with fast training and realtime rendering; unfortunately, the memory requirements of this method for storing and transmission are unreasonably high. We first analyze the reasons for this, identifying three main areas where storage can be reduced: the number of 3D Gaussian primitives used to represent a scene, the number of coefficients for the spherical harmonics used to represent directional radiance, and the precision required to store Gaussian primitive attributes. We present a solution to each of these issues. First, we propose an efficient, resolution-aware primitive pruning approach, reducing the primitive count by half. Second, we introduce an adaptive adjustment method to choose the number of coefficients used to represent directional radiance for each Gaussian primitive, and finally a codebook-based quantization method, together with a half-float representation for further memory reduction. Taken together, these three components result in a x27 reduction in overall size on disk on the standard datasets we tested, along with a x1.7 speedup in rendering speed. We demonstrate our method on standard datasets and show how our solution results in significantly reduced download times when using the method on a mobile device (see Fig. 1).

Additional Files and Images

Weblinks

• Entry in reposiTUm (TU Wien Publication Database)
• DOI: 10.1145/3651282

BibTeX

@article{papantonakis-2024-rmf,
  title =      "Reducing the Memory Footprint of 3D Gaussian Splatting",
  author =     "Panagiotis Papantonakis and Georgios Kopanas and Bernhard
               Kerbl and Alexandre Lanvin and George Drettakis",
  year =       "2024",
  abstract =   "3D Gaussian splatting provides excellent visual quality for
               novel view synthesis, with fast training and realtime
               rendering; unfortunately, the memory requirements of this
               method for storing and transmission are unreasonably high.
               We first analyze the reasons for this, identifying three
               main areas where storage can be reduced: the number of 3D
               Gaussian primitives used to represent a scene, the number of
               coefficients for the spherical harmonics used to represent
               directional radiance, and the precision required to store
               Gaussian primitive attributes. We present a solution to each
               of these issues. First, we propose an efficient,
               resolution-aware primitive pruning approach, reducing the
               primitive count by half. Second, we introduce an adaptive
               adjustment method to choose the number of coefficients used
               to represent directional radiance for each Gaussian
               primitive, and finally a codebook-based quantization method,
               together with a half-float representation for further memory
               reduction. Taken together, these three components result in
               a x27 reduction in overall size on disk on the standard
               datasets we tested, along with a x1.7 speedup in rendering
               speed. We demonstrate our method on standard datasets and
               show how our solution results in significantly reduced
               download times when using the method on a mobile device (see
               Fig. 1).",
  month =      may,
  articleno =  "16",
  doi =        "10.1145/3651282",
  issn =       "2577-6193",
  journal =    "Proceedings of the ACM on Computer Graphics and Interactive
               Techniques",
  number =     "1",
  pages =      "17",
  volume =     "7",
  publisher =  "Association for Computing Machinery (ACM)",
  pages =      "1--17",
  keywords =   "3D gaussian splatting, memory reduction, novel view
               synthesis, radiance fields",
  URL =        "https://www.cg.tuwien.ac.at/research/publications/2024/papantonakis-2024-rmf/",
}