A few months ago I came across maximum mean discrepancy as a measure of distribution difference, and today I read this term and totally forgot what is means and had to find a youtube video to refresh my understanding. This happens a lot of times in my research. I feel like unless something is really basic (e.g. CNN, cross entropy, etc) and used a lot in my day-to-day model building, I easily forgot what I have read. I wonder is it just because I have a bad memory or I do not have a good way to organize information?
i just take notes.
Repetition and working with them. I hope you’re not under the impression that reading a paper once is going to help you remember it. I have to read a paper at least 3-4 times before I feel like I actually really understand it.
I remember reading somewhere that people are only able to retain 10-15% of the information they read in the first go or something.
Honestly, this makes me sad. I completely agree with you and experience this myself but it is disheartening.
I’m currently way too overworked to do this but I have had the idea of making a personal wiki. Read a paper those bazillion times I need until I understand it (some great papers are truly poorly written) and make succinct notes (definitions, math, code, conclusions, usage cases). Then use the wiki whenever I forget something.
I don’t know what platform / tool to use for the wiki because I don’t want to be forever required to pay for a subscription fee yet I want it accessible from multiple devices and have a nice search.
Don’t be sad, it’s just a part of how things are you just have to choose a method and stick to it.
I personally use Notion. I’ve created a database and added properties like date, venue, authors, organizations, etc.
For example, the other day I needed to recap what the BLIP paper was about so I just searched the paper in the database and took a look at the page. On that page I’ve highlighted different text with different colors depending on when I came back to read it.
Took me a while to get this working and into the habit of it though.
I’ve been in the industry for 5 years. I don’t know how many papers I have read but I know that I recall only few. I regret not writing things down earlier but I never have the time. Ugh, I just need to swallow this up and start to do this
Well I don’t know your situation but I feel like the “never have time” excuse may not necessarily be true. Even creating a page in Notion and writing down one line is enough for me. I feel like what was holding me back before was the trap of perfectionism. I wouldn’t want to write anything unless I could make it into some conference-poster-quality page.
It probably isn’t, perfectionism does sound more like it. Thanks!
You guys remember stuff?
In my experience I think it suffices to have a mental index of the high level intuitions of stuff as well as how they relate to each other (unless you’re interested in making innovations in some particular technique in which case you obviously have to remember every details…)
That’s the neat thing. You don’t.
write them down
The only paper I remember how it works is when I try to implement it myself
Or you try to teach someone else
By implementing a model without looking at the paper, you essentially perform autoencoding/masked language modelling and learn a more compact latent representation.
yeah that works for me, but there are too many papers out there
Try a hyperbolic time chamber.
As time passes, you’ll naturally come to understand and remember them. Trying to memorize everything from the start can make it easier to forget.
Make Anki cards and repeat them every day. You’d be surprised how much your brain can retain and use if you revise regularly.
There are lots of engineering tricks given fancy names in ML. No point in memorizing all of them, but keep reading them to maintain intuition.
You define smart or ask chat gpt…
I use Obsidian to note things with markdown + highlight things in the paper + eventually make slides about it
I think about writing blog posts about some methods on my web page to further my understanding of some techniques.
I do the same thing but with Notion. I try to recollect the most key points of a paper, book, etc. and put in the text key words to find them with Ctrl + F and make my own Wiki. Also I use some LLM to resume the content, correct them and stuffs like that.
Does notion comes with Zotero integration ? I use Obsidian because it’s free + I can easily import a paper from my collections with a template which contains the paper’s name, authors etc. I like the community plugins too, makes everything pretty convenient.
I have an overleaf doc of papers I like/are useful. It is linked so I can quickly access each paper.
do you read it e.g. every few days, to refresh your memory? Or just use it as reference? If it is latter, how does it compare with direct Google search?
I use it as reference. It’s much better than google search, because in each section I include only important details from each paper (not the full thing). This lets me quickly remember important stuff about each paper as well as how it connects to relevant work
u/wedbleiz or anyone:
I 'd hugely appreciate seeing a compare between Obsidian and Notion to solve the problem that u/Illustrious-Pay-7516 described.
I like to imagine one research in terms of another. For example I see Luna as a cousin of RMT (core idea of both is to get smaller sequence from a bigger, but methods and goals are very different), but if you squint, you will see the similarities. Helps with breaking down whole paper to smaller parts and see how one research is different from another and how they are similar. And I reward myself with a cookie if I find similarities when papers do not mention each other. I also have a (paper) notebook where I write down notes
Disclaimer, I’m not student/researcher, but a dirty hobbyist
All of us are like this. There are deep reading technics that help slightly but unless you use said method in your day-to-day, you are going to start forgetting things eventually.