There was some hype around neural processes which seem like a marriage between non-parametric models like GPs and NN but have not heard anything about them recently.
Have they been superseded by something else? I have some spatio temporal data that I am looking to play with and wonder if this is a good way to dive into them but want to convince myself that they are still a somewhat relevant family of models to spend time on.
You must log in or register to comment.
I don’t know if neural processes were superceeded, but I have used them in work before successfully. I think there are other approaches to combining ideas from GPs with neural networks, not sure what the state of things is right now.
Last I checked there was a family of similar models so you should look there, particularly around the few shot / meta-learning problems that neural processes were created for.
I have worked (academia & industry) with both GPs and NNs, and never even heard of neural proceses, so… not really.
For spatio-temporal data GNNs are the hot topic and more known approach
I am not doing working with INLA-SPDE but it is great for spatial analysis.
Thanks. Never heard of this but will look into it. My R skills are very dodgy but will try and understand.
Even though I work in developing alternate methods, I have to admit that their tutorials and vignettes are great. They also have a super reactive helpdesk. And of course, you have nice interpretable Bayesian hierarchical models, with great computational performance. If you want to noodle around with your data, give it a shot. I’m always praying them I should just work with them LOL
I will :) It is more of a learning exercise for me and this seems just like the kind of stuff I am looking for. There are many things in there that I probably do not know much about and it is a good learning opportunity :)
Cool ! By the way what kind of data do you have ? I actually work on spatio-temporal data
It is mostly traffic flow data across the UK highways. I am trying to use it to predict traffic flows. I have seen some tutorials using this with GNNs as well.
Also interested in adding weather and event data and other things that might affect flow. So looking into solutions that can handle multivariate data, deal with missing values and also be able to handle discrete/categorical as well as continuous feature variables.
Never heard of neural processes. If you mean deep architectures based on Gaussian Processes (such as Deep GPs or Deep Kernel Learning), does are very much SotA in applied AI for information-restricted domains or in scenarios where you really need a proper uncertainty treatment (such as medical trials, investment banking/corporate finance, datacenter resource allocation and webpage optimization to name a few).
But I am not sure if that answers your question
It was from a paper from Deep Mind in 2018, Ganelo et al.
It’s weird that so many folks in the comments worked with GPs and NNs and never heard of neural processes. They were a big deal until a few years ago: https://yanndubs.github.io/Neural-Process-Family/text/Intro.html
Here is the issue with neural processes: they suck, they really do, on any reasonable real-world problem beyond the simple examples with tons of training data in relatively simple domains. Source: a frustrated grad student who spent hours making conditional neural processes work on a real-world problem.
Thank you. Good to hear about your experience.
So GNNs are the alternative or do you know about something else which might be interesting to look at?