GANs are implicit probabilistic models. This means that they do not learn the distribution of data but rather a mapping from a known distribution (standard Gaussian) to the data distribution. As a result, there is no density estimate because it isn’t modeling the density.

VAEs, on the other hand, can approximate the density indirectly, since they also do not learn the distribution of the data directly. Rather, it learns an encoder which estimates p(z|x) and a decoder p(x|z). However, using simple probabilistic rules, we can derive p(x) = integral of p(x,z) over z. We break down p(x,z) to p(x|z)p(z). We approximate integral of p(x|z)p(z) dz with the monte carlo approximation via sampling to arrive at an estimate of p(x).

If you look at the review: https://arxiv.org/pdf/2103.04922.pdf in Table 1, you’ll see in the rightmost column GANs don’t have any “NLL” - this stands for the negative log likelihood, or if you like the model’s density fit over the distribution. Other classes of models, like VAEs, only give bounds on the density (approximate densities). Flows and autoregressive token predictors can give exact densities. The discriminator of GAN just estimates if something is real or fake, not estimating true probabilities (densities). Adversarial training can, however, be used for anomaly detection (and works quite well, e.g. GANomoly and successors).

GANs indeed learn to generate samples from the data distribution.

VAEs learn how to encode samples to parameters (mean, variance) of a latent-distribution. De VAE-decoder then maps samples from that latent-distribution back to input samples. If basically an autoencoder that tries to do “input->code->reconstructed input”, but with the code being a compact probability distribution instead of a point.
You can use a VAE as an outlier detectors by looking at the reconstruction error. If you e.g. have trained a cat-image VAE then if will ouput cat images. You can generate random code samples and run those through the decoder sttep and then you’ll get random cat pictures. If you feed it a cat picture, encode it, and then decode it, you get something similar to your original input cat image out again. This is because it is an auto-encoder. The reconstruction error is small in this case. If you however feed it a dog image, then the encoder will try to map it to a cat-code, however, the decoder will then still always generate a cat image. In this case the input dog image and the ouput cat image will have a larger distance / reconstruction error.
There is yet another type of generative models called “flow models” that explicitely model the data density. Flow models use invertible function and allow you to evaluate the pdf directy, whereas VAEs only tell you how well it can auto-encode a sample, and it will be trained to do that (only) well for samples from the trainset.

GANs learn to generate samples in similar ratios as the original data: if there’s 10% dogs, there will be 10% dogs in the samples. But they don’t work backwards from a dog image to 10%, you might say - they are ‘likelihood-free’. They just generate plausible images. They don’t know how plausible an existing image is.

In theory, a VAE can tell you this and look at a dog image and say ‘10% likelihood’ and look at a weird pseudoimage and say ‘wtf this is like, 0.00000001% likely’, and you could use it to eliminate all your pseudoimages. In practice, they don’t always work that well for outlier detection and seem to be fragile. So, the advantage of VAEs there may be less compelling than it sounds on a slide.

She might be refering to the findings in this paper: https://proceedings.neurips.cc/paper_files/paper/2019/file/959ab9a0695c467e7caf75431a872e5c-Paper.pdf

GANs are not supposed to do that, all they do is use surfaces. If you want 3D GAN you are going to have to create it

I learned GAN recently too so take it with a grain of salt.

The generative network learns a function that takes random noise as an input and returns a generated sample. As a consequence, the network learns the distribution of the true samples, but that information is hard to retrieve because it is encoded as weights of neural networks. So yes, it learns the distribution, but we cannot use it because it is in a hard-to-use format.

there are too many people in the city that can’t seem to see the problem.