Locality sensitive hashing gets you fast multidimensional retrieval on traditional databases, so vector databases aren’t important unless you need to detect similarity in feature space. I thought vector databases were important until I learned about LSH because I assumed high dimensional retrieval was slow and not exploiting concentration of measure.

Why does gradient descent have good inductive biases? Do the inductive biases of non gradient based optimizers differ?

I’m looking into a problem in this area now. I’m currently looking at the paper Equivariant Neural Rendering, but it doesn’t seem very sophisticated. Can you recommend any better geometrical approaches to the novel view synthesis problem? Over the past few days I have been reading a lot by Hinton about how CNNs are bad at geometry, but his own preferred solution of Capsule Networks doesn’t seem to scale very well.