Inhalt

•  
   Introduction
•  
   Contributions and Outline
•  
  Deep Generative Models
•  
  Deep Neural Network
•  
   The Structure of Deep Neural Networks
•  
   Learning Objectives
•  
   Training Deep Neural Networks
•  
   Further Remarks on Training Deep Neural Networks
•  
  Generative Models
•  
   Generative Model Framework and Learning
•  
   Graphical Model
•  
   Training of Generative Models with Latent Variables
•  
  Linking Deep Neural Networks and Generative Models
•  
   Deep Generative Model
•  
   Variational Autoencoder (VAE)
•  
   Conditional Variational Autoencoder (CVAE)
•  
   Further Remarks on VAEs
•  
   Amortized VAE
•  
   Blurry Reconstruction
•  
   Disentangled Latent Space and Generative Factors
•  
   VAE for Semi-Supervised Learning
•  
  Multi-Modal Perception
•  
   Multi-Modal Machine Learning – Definition and Taxonomy
•  
  Multi-Modal Properties
•  
   Heterogeneity of Multi-Modal Data
•  
   Domain
•  
   Task
•  
   Conclusion
•  
   Correlations between Modalities, Classes, and Attributes
•  
   Requirements for Multi-Modal Data and Observation Ambiguities
•  
   Conclusion and Challenges faced in this Work
•  
  Deep Multi-Modal Machine Learning
•  
   Fusion in Autonomous Architectures
•  
   Deep Multi-Modal Fusion
•  
   Multi-Modal Deep Generative Models
•  
   Requirements
•  
   Literature Overview
•  
   Conclusion and Challenges for this Work
•  
   Multi-Modal Variational Autoencoder
•  
  Preliminary Approaches
•  
  Joint Variational Autoencoder (JVAE)
•  
   Derivation of the Training Objective
•  
   Discussion
•  
  Joint Multi-Modal Variational Autoencoder (JMMVAE)
•  
   Derivation of the Training Objective
•  
   Discussion
•  
   M²VAE
•  
  Derivation of the Bi-Modal M²VAE
•  
   The Variational Lower Bound
•  
   Approximating Inference (i.e., rewriting LM2)
•  
   Discussion
•  
   Extension to three Modalities
•  
   JMMVAE for Three Modalities
•  
   Proposed M²VAE for three Modalities
•  
   Derivation of a General Expression for an Arbitrary Number of Modalities
•  
   Realization as Deep Neural Network
•  
   Conscious vs. Unconscious M²VAE
•  
   Comparison of Uni-Modal and Mixture Distribution
•  
   Uni-Modal versus Mixture Distribution
•  
   Mixture versus Uni-Modal Distribution
•  
   Discussion
•  
   Evaluation of Convexity for Optimization
•  
   Investigation of Derivatives
•  
   Exemplary Demonstration
•  
   Discussion
•  
   Auto Re-Encoding
•  
   In-Place Sensor Fusion
•  
   Training with Re-Encoding
•  
   Re-Encoding Demonstration
•  
  Multi-Modal Data Sets and their Properties
•  
   Review of Available Data Sets
•  
   Data Set Properties
•  
   Multi-Modal Data Sets in the Wild
•  
   2D Object
•  
   3D Object
•  
   2D & 3D Faces
•  
   Human Activity Recognition (HAR)
•  
   Autonomous Ground Vehicle (AGV)
•  
   Simulator
•  
   Proposed Data Sets
•  
   Exclusive OR (XOR)
•  
   Mixture of Gaussians (MoG)
•  
   Camera+LiDAR
•  
   Rubiks
•  
   eMNIST
•  
   Generating true Multi-Modal data sets by Collation
•  
   eMNIST
•  
   Discussion and Choice of Suitable Data Sets
•  
   Metrics, Evaluations, and Results
•  
  Scores and Metrics
•  
   Scores and Metrics in the Wild
•  
   Discussion and Choice of Suitable Metrics
•  
   Results
•  
  Hyperparameter Analysis
•  
   Learning Rate Analysis
•  
   Correlation Analysis
•  
   Ablation Study
•  
   XOR Evaluation
•  
   MoG Evaluation
•  
   Weight-Sharing and Mutual KLD
•  
   Competitive Evaluation and Other Data Sets
•  
   Log-Likelihood and FID
•  
   Downstream Performance and JSD
•  
   Discussion
•  
  Applications
•  
   Active Sensing through Ambiguity
•  
   Embedding M²VAE in a Learning Framework
•  
   Analysis of the Observation–Action Process
•  
   Perceived Environment
•  
   Evaluation
•  
  Rubiks
•  
   The Rubiks data set
•  
   Intrinsic Curiosity Module (ICM) vs. Multi-Modal Variational Autoencoder (M²VAE)
•  
   Results
•  
   Active Sensing with Distributed and Heterogeneous Robots
•  
   Discussion
•  
   Conclusion and Outlook
•  
   Acronyms and Abbreviations
•  
   List of Figures
•  
   List of Tables
•  
   Bibliography
•  
   Most Influential Conferences and Journals
•  
  Supplemental Material
•  
   List of Applied Software
•  
  Architecture Setups and Assets
•  
   VAE Training Setup
•  
   CVAE Training Setup
•  
   Re-Encoding Training Setup
•  
   eMNIST CVAE Training Setup
•  
   Hyperparameter Dependencies Training Setup
•  
   XOR Training Setup
•  
   MoG Training Setup
•  
   Shared Weights Training Setup
•  
   Competitive Evaluation Setup
•  
   Rubiks Data Set Evaluation Setup
•  
   AMiRo-CITrack Evaluation Setup
•  
   Latent Space Statistics
•  
   Semi-Supervised VAE
•  
   List of Data Set Websites
•  
  Alternative nomenclature for the Variational Autoencoder
•  
   The Variational Bound
•  
   Approximate Inference (i.e. rewriting L)
•  
   Derivation for the Joint Multi-Modal VAE via Variation of Information
•  
   Derivation of KLD-Derivative Inequality
•  
   Variation of Information
•  
   Lie Groups
•  
  CITrack & AMiRo
•  
   CITrack
•  
   AMiRo Applications in the CITrack
•  
   AMiRo–CITrack Interaction
•  
   Model Identification
•  
   Data Labeling and Verification
•  
  Mathematical Foundations
•  
   Expected Value of a Random Variable
•  
  Further Quantities of a Random Variable
•  
   Entropy
•  
   Cross Entropy
•  
   Kullback–Leibler Divergence (KLD)
•  
   Jensen–Shannon Divergence (JSD)
•  
   KLD for two Gaussian distributions
•  
   Positiveness of Entropy for a Gaussian Distribution
•  
   Jensen's Inequality
•  
   Mixture of Gaussian versus Gaussian - Derivation via Jensen's Inequality