TIL about the Gell-Mann amnesia effect; when experts find articles published within their field to be full of errors, but trust articles about other fields in the same publication - eviltoast
  • givesomefucks@lemmy.world
    link
    fedilink
    English
    arrow-up
    12
    arrow-down
    4
    ·
    3 months ago

    Eh.

    If you have any familiarity with academic research then you know almost anything can pass peer review.

    Being published doesn’t make something true, you still have to read. And evaluate the article especially the methods.

    • canihasaccount@lemmy.world
      link
      fedilink
      English
      arrow-up
      2
      arrow-down
      1
      ·
      3 months ago

      Journal quality can buffer this by getting better reviewers (MDPI shouldn’t be seen as having peer review at all, but peer review at the best journals–because professors want to say on their merit raise annual evals that they are doing the most service to the field by reviewing at the best journals–is usually good enough at weeding out bad papers), but it gets offset by the institutional prestige of authors when peer-review isn’t double-blind. I’ve seen some garbage published in top journals by folks that are the caliber of Harvard professors (thinking of one in particular) because reviewers use institutional prestige as a heuristic.

      When I’m teaching new grad students, I tell them exactly what you said, with the exception that they can use field-recognized journal quality (not shitty metrics like impact factor) as a relative heuristic until they can evaluate methods for themselves.