Hello.
I am seeing the GAN specialization course on Coursera by instructor Sharon Zhou.
In one of the lectures, she says that a disadvantage of GANs is that it cannot do density estimation and thus is not useful for anomaly detection. (not sure if you can access it).
In the next video, she says that VAEs don’t have this problem.
I am a little confused about this. Could anyone please explain what she means?
As far as I can understand from the lecture, density estimation means learning how probable/frequent particular features are in a dataset. Like, how probable is it that a dog will have droopy ears. Then, we can use this info to detect anomalies if they do not exhibit these features.
But, isn’t this exactly what GANs learn? aren’t GANs learning to mimic the distribution of the training data?
Also, how is a VAE different in this particular regard?
Could someone please help explain this?
Thank you.
GANs are implicit probabilistic models. This means that they do not learn the distribution of data but rather a mapping from a known distribution (standard Gaussian) to the data distribution. As a result, there is no density estimate because it isn’t modeling the density.
VAEs, on the other hand, can approximate the density indirectly, since they also do not learn the distribution of the data directly. Rather, it learns an encoder which estimates p(z|x) and a decoder p(x|z). However, using simple probabilistic rules, we can derive p(x) = integral of p(x,z) over z. We break down p(x,z) to p(x|z)p(z). We approximate integral of p(x|z)p(z) dz with the monte carlo approximation via sampling to arrive at an estimate of p(x).