I’ve been diving a lot deeper into some interesting neural network papers recently and I’m looking to try and implement some of the models detailed in the papers. In general, I know that many papers include the code or I can just google the code to implement the model but I want to push myself to start implementing from scratch more.

Could anyone offer some tips on how they got started or gained the skills to be able to implement a model effectively within a few hours? Any advice would be much appreciated!

  • DollyNorman@alien.topBEnglish
    1·
    1 year ago

    I’ve found https://paperswithcode.com/ and github search & topics useful tools. In my experience implementing a paper from scratch is a fantastic way to gain a deeper understand of a paper. Don’t be discouraged, I can’t remember a single paper I’ve tried to implement that provided all the details necessary to implement.

      • DollyNorman@alien.topBEnglish
        1·
        1 year ago

        I think this is why implementing is such a useful learning tool. Papers I tried to implement I probably read cover-to-cover 10 or 20 times, as opposed just skimming abstract, method, results.

        When missing key info, after searching the paper a few times:

        • my first step was see if I could find anything on paperswithcode or github.
        • failing that, google searches. see if I could find anything on forums, stack exchange sites, reddit.
        • uni library or academic paper web search engines
        • see if any papers that cite the paper I’m implementing give some clue
        • last resort was to look into papers they cite.

        That was all the tricks I had, keen to know any more.

        • Ifkaluva@alien.topBEnglish
          1·
          1 year ago

          What do you do if you find a reference implementation? Just run it? Try to implement from scratch?

          • DollyNorman@alien.topBEnglish
            1·
            1 year ago

            I tried to copy as much as possible from the paper, then fill in the blanks with how I would have solved it. I only had a partial solution but I still learnt heaps more than just reading the paper.

        • DryArmPits@alien.topBEnglish
          1·
          1 year ago

          You can also contact the authors to ask them. Most of us are not monsters and will happily talk about our work. :)

  • Nique_Eau@alien.topBEnglish
    1·
    1 year ago

    i found it helpful to read related papers and code, break down the model into smaller components, and slowly build it up. sometimes you just gotta experiment and fill in the gaps as you go.

  • brainx98@alien.topBEnglish
    1·
    1 year ago

    I am also in that posture. i will be reading the comments. what i am currently doing is buildong from scratch most of the common Models.