Close Menu
Chicago News Journal
    Facebook X (Twitter) Instagram
    • Contact us
    • About us
    • Amazon Disclaimer
    • DMCA / Copyrights Disclaimer
    • Privacy Policy
    • Terms and Conditions
    Facebook X (Twitter) Instagram YouTube TikTok
    Chicago News JournalChicago News Journal
    • Home
    • US News
    • Politics
    • Business
    • Science
    • Technology
    • LifeStyle
    • Music
    • Television
    • Film
    • Books
    • Contact
      • About us
      • Amazon Disclaimer
      • DMCA / Copyrights Disclaimer
      • Privacy Policy
      • Terms and Conditions
    Chicago News Journal
    Home»Science

    Theorists are good peer reviewers – but tend to prefer significance over rigour, study finds

    AdminBy AdminJuly 23, 2023 Science
    Facebook Twitter Pinterest LinkedIn Tumblr Email Reddit Telegram


    Accept/reject
    Key metrics: A study find that peer reviewers can often judge the originality, rigour and significance of new work consistently — at least in theoretical physics (courtesy: Shutterstock/Nixx-Photography)

    Theoretical physicists do a good job when peer reviewing scientific papers, but tend to be more impressed by the significance of new research rather than the rigour with which it was carried out. That is the provocative suggestion of a new study carried out by researchers in the UK and Poland that examines the reliability of peer review in theoretical physics.

    Peer review involves sending research papers to independent, external referees to decide if an article is scientifically credible and appropriate for the journal to which it has been submitted. Despite being a central part of science for hundreds of years, few academics are trained in peer review, and anecdotal evidence suggests that disagreements between individual reviewers about a paper are common.

    “In an ideal world, all reviewers would evaluate each submission with perfect accuracy and render a perfect decision,” says Mike Thelwall, a data scientist from the University of Sheffield, who carried out the study. But given that no work can be perfect in all of the metrics used to judge a paper’s quality, reviewers make what Thelwall dubs “a judgement call” about the extent to which an article achieves an acceptable level.

    The three core components of quality used to judge a paper are originality, rigour and significance. However, Thelwall says that no-one had previously checked if reviewers judge these metrics effectively, even though they are, for example, part of the guidelines for assessors in the UK’s Research Excellence Framework.

    Decision makers

    Thelwall and physicist Janusz Hołyst of the Warsaw University of Technology therefore analysed reviewer scores in 505 theory papers published in SciPost Physics — a journal that practises open peer review. The duo chose to focus on theoretical physics because they say it is best-case-scenario for peer review, in that reviewers should be more likely to understand all aspects of a given submission than in other areas of science.

    Each paper in the study had at least two scoring reviewers and the analysis revealed a “moderate” degree of agreement between reviewers on all three key metrics. “The results should cause no worries at all for SciPost Physics — their reviewers are doing an excellent job,” Thelwall says.

    However, he worries that journals publishing research in fields where it is harder for individual reviewers to make a sound judgement might have much lower agreement rates for reviewers. “By the law of averages, [they may] tend to make bad decisions moderately often due to an unlucky choice of reviewers,” he says. The authors suggest such issue might be minimized via editorial oversight, better guidelines and improved reviewer training.

    Illustration of a computer screen and a magnifying glass

    Peer review under the spotlight


    The study also looked at which of the quality measures tend to be ranked the same by individual reviewers. The results suggest that — in theoretical physics at least — referees tend to rank rigour lower. This means it could be harder to publish what Thelwall calls a “highly rigorous original study”, but easier to get a “significant original study” accepted.

    In fact, theoretical physicists striving to create significant work might benefit from focusing on original ideas rather than generating exhaustive evidence to support their work, the authors suggest. “Of course, we need all types of research and if everyone gives up on rigorous ideas it will be a disaster for science,” cautions Thelwall.

    Stefan Thurner, a physicist from the Medical University of Vienna who was not involved in the study but has carried out studies into peer review, says the study is interesting. “The raw data indicate that the reviewers are remarkably consistent in practically all categories,” he says. “In more than 400 out of 505 cases, they differ not at all or by one score unit. I am sure that many scientists have the subjective feeling that referee reports differ more often than that.”

    Thurner adds, however, that future research could compare the results with, say, reference models in which papers were randomly assigned to referees.

    Share. Facebook Twitter Pinterest LinkedIn Tumblr Email Reddit Telegram

    You might also be interested in...

    ZWO Seestar S30 Plus review

    November 10, 2026

    'The first species we might be able to communicate with, interact with vocally, might be a bird': An animal researcher explains how she decoded birdsong

    November 10, 2026

    Highly active Russian volcano plays peek-a-boo with an astronaut

    November 10, 2026

    Iran war helps fuel global drop in fossil fuel emissions — but theres a catch

    November 10, 2026

    Nazca infant had rare, fatal birth defect that left her with one central eye 1,600 years ago

    November 9, 2026

    Did human sacrifice help lower violence? New research in Peru leaves intriguing question

    November 9, 2026
    Popular Posts

    Interview with Aletheia von Gottlieb, Author of Love Off the Ice (Sweet Snowfall Kisses Book 1) – NewInBooks

    What Were Wearing: Friendsgiving

    In historic move, House Democrats nominate Hakeem Jeffries for speaker of the House

    Book Riots Deals of the Day for November 18, 2026

    Voyager 1 at one light-day: Historian revisits Golden Record's forgotten details

    DWTS Fans Discover Sneaky Loophole To Rig Votes For Favorite

    Categories
    • Books (2,456)
    • Business (3,449)
    • Events (34)
    • Film (1,062)
    • LifeStyle (2,922)
    • Music (2,796)
    • Politics (2,355)
    • Science (1,951)
    • Technology (1,817)
    • Television (3,628)
    • US News (3,293)
    Archives
    Useful Links
    • Contact us
    • About us
    • Amazon Disclaimer
    • DMCA / Copyrights Disclaimer
    • Privacy Policy
    • Terms and Conditions
    Popular Posts
    • The Book News We Covered This WeekThe Book News We Covered This Week
    • Jared Padalecki Fire Country Spinoff News Makes Us Wonder How Much Abuse This Franchise Can TakeJared Padalecki Fire Country Spinoff News Makes Us Wonder How Much Abuse This Franchise Can Take
    • First glimpse of sperm whale birth reveals teamwork to support newbornFirst glimpse of sperm whale birth reveals teamwork to support newborn
    • Jason Clarke Considering a Role in ‘Heat 2’Jason Clarke Considering a Role in ‘Heat 2’
    • How Jenn Tran’s Relationship With Jonathon Changed After ShowHow Jenn Tran’s Relationship With Jonathon Changed After Show
    Archives
    Categories
    • Books
    • Business
    • Events
    • Film
    • LifeStyle
    • Music
    • Politics
    • Science
    • Technology
    • Television
    • US News
    Facebook X (Twitter) Instagram YouTube TikTok
    © 2026 Chicago News Journal. All rights reserved. All articles, images, product names, logos, and brands are property of their respective owners. All company, product and service names used in this website are for identification purposes only. Use of these names, logos, and brands does not imply endorsement unless specified. By using this site, you agree to the Terms of Use and Privacy Policy.

    Type above and press Enter to search. Press Esc to cancel.