Visual Analytics for Convolutional Neural Network Robustness

Information

• Publication Type: Master Thesis
• Workgroup(s)/Project(s):
  • Visualization Group
• Date: April 2022
• Date (Start): 2019
• Date (End): April 2022
• Diploma Examination: May 2022
• First Supervisor: Manuela Waldner

Abstract

Convolutional neural networks (CNNs) are a type of machine learning model that is widely used for computer vision tasks. Despite their high performance, the robustness of CNNs is often weak. A model trained for image classification might misclassify an image when it is slightly rotated, blurred, or after a change in color saturation. Moreover, CNNs are vulnerable to so-called “adversarial attacks”, methods where analytically computed perturbations are generated which fool the classifier despite being imperceptible by humans. Various training methods have been designed to increase robustness in CNNs.

In this thesis, we investigate CNN robustness with two approaches: First, we visualize differences between standard and robust training methods. For this, we use feature visualization, a method to visualize the patterns which individual units of a CNN respond to. Subsequently, we present an interactive visual analytics application which lets the user manipulate a 3d scene while simultaneously observing a CNN’s prediction, as well as intermediate neuron activations. To be able to compare standard and robustly trained models, the application allows simultaneously observing two models. To test the usefulness of our application, we conducted five case studies with machine learning experts. During these case studies and our own experiments, several novel insights about robustly trained models were made, three of which we verified quantitatively. Despite its ability to probe two high performing CNNs in real-time, our tool fully runs client-side in a standard web-browser and can be served as a static website, without requiring a powerful backend server

Additional Files and Images

Weblinks

• Computer Graphics Forum paper
  Journal paper associated with the thesis
• online playground
  Online playground
• Entry in reposiTUm (TU Wien Publication Database)

BibTeX

@mastersthesis{sietzen-2022-vacnnr,
  title =      "Visual Analytics for Convolutional Neural Network Robustness",
  author =     "Stefan Sietzen",
  year =       "2022",
  abstract =   "Convolutional neural networks (CNNs) are a type of machine
               learning model that is widely used for computer vision
               tasks. Despite their high performance, the robustness of
               CNNs is often weak. A model trained for image classification
               might misclassify an image when it is slightly rotated,
               blurred, or after a change in color saturation. Moreover,
               CNNs are vulnerable to so-called “adversarial attacks”,
               methods where analytically computed perturbations are
               generated which fool the classifier despite being
               imperceptible by humans. Various training methods have been
               designed to increase robustness in CNNs.  In this thesis, we
               investigate CNN robustness with two approaches: First, we
               visualize differences between standard and robust training
               methods. For this, we use feature visualization, a method to
               visualize the patterns which individual units of a CNN
               respond to. Subsequently, we present an interactive visual
               analytics application which lets the user manipulate a 3d
               scene while simultaneously observing a CNN’s prediction,
               as well as intermediate neuron activations. To be able to
               compare standard and robustly trained models, the
               application allows simultaneously observing two models. To
               test the usefulness of our application, we conducted five
               case studies with machine learning experts. During these
               case studies and our own experiments, several novel insights
               about robustly trained models were made, three of which we
               verified quantitatively. Despite its ability to probe two
               high performing CNNs in real-time, our tool fully runs
               client-side in a standard web-browser and can be served as a
               static website, without requiring a powerful backend server",
  month =      apr,
  address =    "Favoritenstrasse 9-11/E193-02, A-1040 Vienna, Austria",
  school =     "Research Unit of Computer Graphics, Institute of Visual
               Computing and Human-Centered Technology, Faculty of
               Informatics, TU Wien",
  URL =        "https://www.cg.tuwien.ac.at/research/publications/2022/sietzen-2022-vacnnr/",
}