Reproducibility, Verification, and Validation of Experiments on the Marschner-Lobb Test Signal

Information

• Publication Type: WorkshopTalk
• Workgroup(s)/Project(s):
  • Visualization Group
• Date: 25. May 2015
• Event: EuroVis Workshop on Reproducibility, Verification, and Validation in Visualization (EuroRV3)
• Lecturer: Viktor Vad
• Location: Cagliari, Sardinia, Italy
• Journal: @inproceedings {eurorv3.20151140
• Pages (from): 1
• Pages (to): 3
• Workshop date: 25. May 2015 – 29. May 2015

Abstract

The Marschner-Lobb (ML) test signal has been used for two decades to evaluate the visual quality of different volumetric reconstruction schemes. Previously, the reproduction of these experiments was very simple, as the ML signal was used to evaluate only compact filters applied on the traditional Cartesian lattice. As the Cartesian lattice is separable, it is easy to implement these filters as separable tensor-product extensions of well-known 1D filter kernels. Recently, however, non-separable reconstruction filters have received increased attention that are much more difficult to implement than the traditional tensor-product filters. Even if these are piecewise polynomial filters, the space partitions of the polynomial pieces are geometrically rather complicated. Therefore, the reproduction of the ML experiments is getting more and more difficult. Recently, we reproduced a previously published ML experiment for comparing Cartesian Cubic (CC), Body-Centered Cubic (BCC), and Face-Centered Cubic (FCC) lattices in terms of prealiasing. We recognized that the previously applied settings were biased and gave an undue advantage to the FCC-sampled ML representation. This result clearly shows that reproducibility, verification, and validation of the ML experiments is of crucial importance as the ML signal is the most frequently used benchmark for demonstrating the superiority of a reconstruction scheme or volume representations on non-Cartesian lattices.

Additional Files and Images

Weblinks

• https://diglib.eg.org/handle/10.2312/eurorv3.20151140.001-003

BibTeX

@WorkshopTalk{Vad_Viktor_2015_RVV,
  title =      "Reproducibility, Verification, and Validation of Experiments
               on the Marschner-Lobb Test Signal",
  author =     "Viktor Vad and Bal\'{a}zs Cs\'{e}bfalvi and Peter Rautek and
               Eduard Gr\"{o}ller",
  year =       "2015",
  abstract =   "The Marschner-Lobb (ML) test signal has been used for two
               decades to evaluate the visual quality of different
               volumetric reconstruction schemes. Previously, the
               reproduction of these experiments was very simple, as the ML
               signal was used to evaluate only compact filters applied on
               the traditional Cartesian lattice. As the Cartesian lattice
               is separable, it is easy to implement these filters as
               separable tensor-product extensions of well-known 1D filter
               kernels. Recently, however, non-separable reconstruction
               filters have received increased attention that are much more
               difficult to implement than the traditional tensor-product
               filters. Even if these are piecewise polynomial filters, the
               space partitions of the polynomial pieces are geometrically
               rather complicated. Therefore, the reproduction of the ML
               experiments is getting more and more difficult. Recently, we
               reproduced a previously published ML experiment for
               comparing Cartesian Cubic (CC), Body-Centered Cubic (BCC),
               and Face-Centered Cubic (FCC) lattices in terms of
               prealiasing. We recognized that the previously applied
               settings were biased and gave an undue advantage to the
               FCC-sampled ML representation. This result clearly shows
               that reproducibility, verification, and validation of the ML
               experiments is of crucial importance as the ML signal is the
               most frequently used benchmark for demonstrating the
               superiority of a reconstruction scheme or volume
               representations on non-Cartesian lattices.",
  month =      may,
  event =      "EuroVis Workshop on Reproducibility, Verification, and
               Validation in Visualization (EuroRV3)",
  location =   "Cagliari, Sardinia, Italy",
  journal =    "@inproceedings {eurorv3.20151140",
  URL =        "https://www.cg.tuwien.ac.at/research/publications/2015/Vad_Viktor_2015_RVV/",
}