by Danielle Gottesman and Kate Bernhardt
Stephen Chaudoin of Harvard’s Department of Government, lamenting the traditional academic journal publishing process—which can take years—notes that “you only get so many bites of the apple with journals.” But Chaudoin is finding that IQSS’s Magaro Peer Pre-Review Program, or PPR, is dramatically changing the calculus by improving papers before they are ever submitted to journals.
Chaudoin first used PPR in 2021 to bridge a gap in the publication process and improve the chances that his research would be disseminated. Within a few short weeks—the amount of time it took to receive his first round of feedback from a wider community of peers outside his immediate network—he quickly came to see the PPR process, as well as the program itself, as “a way to get well-positioned to be published” in a competitive space where “journals are having trouble getting reviewers.”
“A review for the journal proper,” he said, “has high stakes, [and] a decision has to be made on whether or not to publish. [The reviewers] are understandably more critical and focus on problems associated with publishing immediately.” Unlike what Chaudoin had previously experienced with journals, the metrics for the Peer Pre-Review Program, he found, were structured on how to improve the work and retain its strengths, rather than whether it should be greenlit for publication.
“It can take a month alone just for a desk review [from a journal], where you’re waiting, not getting your research out there,” says Steven Worthington, director of the Data Science Services team and a lead scientist at IQSS. “This dance can take anywhere from two months to a year to get a first set of reviews, then six months to three years for the whole process. If you submit a paper and it takes six months to get feedback, it can take another six for the next round of reviews, versus [with PPR] two weeks and skipped rounds of reviews.”
Few submissions, including those from top scholars, are accepted on the first round, and an author typically receives a journal rejection that lacks sufficient clarity on how to make the piece stronger or more apt to reach its intended audience. It may then be offered to another journal, but without thorough commentary from reviewers, the chance for improvement is limited. The developers of PPR saw an opportunity: could connecting scholars with peer reviewers before a paper is in its final form result in better papers, higher publication rates, and superior access to cutting-edge information across and potentially beyond academia?
“I don’t think anyone has tried to do this or identified it as something that’s missing from the current journal review process” says Steve Worthington. “Researchers want to know, Why go through this process, which adds an extra month to my overall time? The answer has always been: You can pick up on extra feedback in this review that will potentially save you many, many months and give you targeted feedback—even on grants—much earlier; in 30-35 days. The reviews are more constructive, more timely, and they can save you from rewriting essentially whole chunks or whole papers.”
Prompt, Comprehensive Feedback
When peer pre-review participant Kosuke Imai joined Harvard as Professor of Government and of Statistics in 2018, he became one of the first researchers to take advantage of the Peer Pre-Review Program, by seeking help on a paper that addressed a subject that was fairly novel to him.
“I wanted to get feedback from those who were more experienced in this subject before I sent the paper out to a journal—really, into the world,” he said. Known for his work at Princeton and at Tokyo University, Imai specializes in algorithms as they apply to social science research. He and his team prepared a scientific paper to submit to a scholarly journal for review and, with hope, publication. Imai’s goal was to provide a foundation upon which a comprehensive array of researchers could build.
Here is where Imai made use of the PPR Program. Normally, when a paper is sent to a journal, its author regards it as finished—meaning it contains polished, strong arguments and considerable evidence for its conclusions. The journal reviews it, then sends it out for peer review. Reviewers may then get held up by unclear language or data, which usually prompts a paper’s return to the journal, followed by a likely rejection to its author(s). The option to make edits, if permitted, typically elicits an entirely new round of reviews, adding considerable time to the process. Professor Imai appreciated the prospect of obtaining prompt, quality input on his work and felt it would stand stronger with the kind of preliminary feedback the PPR program could provide. He set forth eagerly.
“I’ve used the PPR program five or six times now,” says Imai. “Sometimes it only takes one round of comments and suggestions for revisions, but I’ve also had back-and-forth communication where someone reads a paper’s second draft—or additional drafts. The mutual anonymity [makes] it possible for [reviewers] to feel comfortable criticizing the work, and I [get] timely feedback—within a month.” With the additional perspective, Imai was able to rethink and revise until it was journal-ready.
Professor of Government Yuhua Wang has used PPR multiple times, from gauging interest in a book manuscript, which he called “invaluable” to his revision process, to obtaining feedback prior to a journal submission. “I suggested some reviewers and got three back. They were helpful, detailed, and provided friendly, deeply constructive feedback. I made changes and resubmitted. For this particular journal,” he said, “I am sure that the paper would have been rejected out of hand if I hadn’t had [the pre-review].”
Seeing Opportunity in a Challenge
Chase Harrison, associate director of the Harvard Program on Survey Research and Senior Preceptor in Survey Methods in Harvard’s Department of Government, was part of the IQSS team that created and launched the Peer Pre-Review Program. “Science is based on peer review,” says Harrison, “but there’s a big time lag between a working draft paper and one that can be published and cited. We wanted to offer faculty feedback that would improve their research.”
Harrison and the IQSS team created and implemented the Peer Pre-Review Program to provide eligible scholars with a collaborative—and ideally more direct—route to publication. Since its launch in early 2019, more than 60 Harvard social scientists, from PhD graduates to senior and junior faculty, have benefited from the Peer Pre-Review program, impacting time-to-publication, quality of scientific research, and scholarly productivity—all of which constitute the program’s primary objectives.
Steve Worthington feels that with continued momentum, the PPR Program could upend the existing journal model, to the benefit of research worldwide. Academics who have used PPR recognize it as a more linear path to getting published—one they have been seeking for decades—as well as a powerful shift in approach that is poised to influence the larger scientific community.
“Peer pre-reviewers are freer to give broader feedback,” Chaudoin finds. “I get the impression that [they] can let themselves be more positive and balanced to say more about what they do and don’t like, which is not otherwise the case. If the exact same person wrote [the] review for a top journal, it would likely have a different outcome.”
While the program is still young, it has proven its value to the social science community with more than 60 reviews completed for researchers across several disciplines. PPR’s unprecedented approach has demonstrated the potential to disrupt, in a critical way, the traditional method of scholarly publication. The advancement of social science lies at the heart of the Peer Pre-Review Program, serving IQSS’s core mission of bringing together researchers across disciplines to develop strong, creative solutions to humanity’s most challenging issues.
“Researchers are already sending work to friends and colleagues in the field, but if you’re early in your career,” Worthington adds, “you don’t necessarily have that network. [PPR] fills the gap between what senior faculty have and junior faculty do not. There’s a huge added-value component, an unvarnished critique that helps put them at less reputational risk. Tenure packages can often rely on top tier publications, and journal reviews could become more like PPR—it’s the ideal journal model from a researcher perspective.” He adds that the hope is to keep PPR around in perpetuity and eventually see it serve as a pilot for other institutions.
“Every working paper I’ve got, I would think about sending to PPR,” Chaudoin says. “My coauthors are either Harvard graduates, affiliates, or recent PhDs, and they are also on board. I can’t think of anything I would change about the peer pre-review process, and my only advice is to use it.”
You can find more information on the Alexander and Diviya Magaro Peer Pre-Review Program, including how to participate, at the PPR program page.