I started my PhD in NLP a year or so before the advent of Transformers, and finished it just as ChatGPT was unveiled (literally defended a week before). Halfway through, I felt the sudden acceleration of NLP, where there was so much everywhere all at once. Before, knowing one’s domain, and the state-of-the-art GCN, CNN or Bert architectures, was enough.
Since, I’ve been working in a semi-related area (computer assisted humanities) as a data engineer/software developer/ML engineer (it’s a small team so many hats). Not much in terms of latest news, so I tried recently to get up to speed with the recent developments.
But there are so many ! Everywhere. Even just in NLP, not considering all the other fields such as reinforcement learning, computer vision, all the fundamentals of ML etc. It is damn near impossible to gather an in-depth understanding of a model as they are so complex, and numerous. All of them are built on top of other ones, so you also need to read up on those to understand anything. I follow some people on LinkedIn who just give new names every week or so. Going to look for papers in top conferences is also daunting as there is no guarantee that a paper with an award will translate to an actual system, while companies churn out new architectures without the research paper/methodology being made public. It’s overwhelming.
So I guess my question is two fold : how does one get up to speed after a year of not being too much in the field ? And how does one keep up after that ?
I spend an hour each morning scanning the preprints on arxiv.org, scanning a half dozen or so and selecting perhaps one to save for weekly symposium( I like to have at least one really good paper a week to share).
It’s usually easy to tell if a paper is a follow-up or response to another and if that’s the case,.I might skim those too. These get supplemented with what pops up here and HN which might extend back a few months (higher signal, less noise).
This is enough to feel like I have my finger on the pulse of one topic within ML.
Wow, that is a lot of work. It’s awesome that you manage to have the latest and the pulse of AI as you said. That is the kind of discipline I cannot follow. Just one hour at work in the morning would destroy the rest of my day ^^