ICLR 2024 paper reviews are visible on OpenReview. I thought to create a discussion thread for us to discuss any issue/complain/celebration or anything else.

There is so much noise in the reviews every year. Some good work that the authors are proud of might get a low score because of the noisy system, given that ICLR is growing so large these years. We should keep in mind that the work is still valuable no matter what the score is.

  • Mindless_Lie_6720@alien.topB
    link
    fedilink
    English
    arrow-up
    1
    ·
    2 years ago

    Got initial score of 855 but the reviewers just won’t reply to the rebuttal… I even asked the AC and they sent a reminder for the reviewers but no one will reply :(

    I think this reviewing system really has to change. Some papers get reviewers that are active and actually consider raising the score after looking at the rebuttal, and some get lazy reviewers that never look at the rebuttal and the score remains.

    The reviewers know nothing will happen to them and they will again be selected as a reviewer next year, even if they do ‘minimal work’. So frustrating :(

    • alexfan07@alien.topB
      link
      fedilink
      English
      arrow-up
      1
      ·
      2 years ago

      I don’t even know whether they decided to maintain the score after reading the rebuttal or they just forgot about it. At least there should be a checkbox to ensure that they have read the rebuttal.

    • Altruistic_Milk_6609@alien.topB
      link
      fedilink
      English
      arrow-up
      1
      ·
      2 years ago

      In the same boat, stuck with reviewers who don’t want to engage. Don’t know what to do. Does anyone with advice on how to get a reply from reviewers who are active?

      • AardvarkDowntown5013@alien.topB
        link
        fedilink
        English
        arrow-up
        1
        ·
        2 years ago

        There is basically no way. There are many reasons they might not reply, including being sick or occupied with their own borderline papers. Try to be professional about it: post a reminder and then let the Area Chairs (ACs) handle it. Most ACs are well aware of these types of situations. They will most likely place less weight on those non-engaging reviews. Good luck with your rebuttal!

        • Altruistic_Milk_6609@alien.topB
          link
          fedilink
          English
          arrow-up
          1
          ·
          2 years ago

          That’s great, healthy POV and advice. Thank you.

          Although my thought process was a bit different, that is what I did as well.

    • Professional_Pin3290@alien.topB
      link
      fedilink
      English
      arrow-up
      1
      ·
      2 years ago

      Same. I also did many additional experiments, but complete silence from the reviewers. It really sucks and makes me feel a bit helpless. I’m used to reviewer roulette in general, but it feels even worse in this case because I see other submissions where all 4 reviewers are engaged and raised their score after the author response.

      I would kind of understand disinterest by reviewers if my submission had very bad scores and would be hopeless anyway, but I got 8855 and was very positive that I could at least increase one of the 5s to a 6 with my response.