Over the last few weeks, we have continued to add more Groups to Sciety, slowly but steadily building up and setting the scene on what promises to be a new way of evaluating preprints. These Groups add to broadening the scope of subjects covered on Sciety. Additionally, they bring empowering access to researchers to make their work more open, visible and accessible.
One of the latest Groups we added is ASAPbio Crowd Review. They have a unique crowd-sourced approach to preprint review and the way it is organised. This takes ‘open’ to the next level, creating an in-road for early career researchers to take part in open reviews.
ASAPbio believes that preprint review brings many benefits: valuable feedback for authors, further contextualization of the work for readers, and an opportunity for those who participate to hone reviewing skills and contribute to scientific discourse. Considering that participation in public preprint feedback is still relatively low, ASAPbio is keen to experiment with new ways of encouraging participation from a diverse range of researchers.
The Sciety Team
The approach is based on the Select Crowd Review pioneered by the journal Synlett, a model that encouraged participation by a large group (the crowd), in a format that emulates some of the elements of commentary on social media platforms, as it allows brief inline commenting on specific parts of the paper in addition to providing full comprehensive reviews. Synlett had implemented this approach as part of the confidential review process at the journal. But would this transpose to the public review of preprints? ASAPbio thought it was worth finding out, and thus they started work on the crowd preprint review trial.
Iratxe Puebla said, “Our main goals for this trial are to see if it encourages participation in commentary and review of preprints, and whether it provides a channel for contributing public reviews. We decided to start as a trial we would review and learn from, focused on one specific discipline - cell biology - and a short period of time to keep the commitment by trial participants to a reasonable level - three months, from end of August to end of November.“
“We have tried to keep the workflow for contributing comments on the preprints as simple as possible, and are running the trial using existing available tools to keep technical requirements to a minimum. In brief, we contact the authors of the preprint for their consent to include the paper in the trial. If the authors agree, we circulate the preprint to the crowd, a group of cell biologists who have signed up to participate. Crowd members are invited to comment on the preprint using Hypothes.is annotation during the following seven days. At the end of the commenting period, we synthesize all the comments received into a single review, which we post publicly as a TRiP review on bioRxiv and on Sciety.”
Their call for participation has attracted almost 100 researchers who signed up to be crowd members, the majority of whom are early career researchers - sign up will remain open during the trial so if you are interested, you can still join here.
The trial is already underway, with reviews of four preprints being completed so far. Each week, the organisers circulate preprints for feedback and the group responds with public evaluations which are then made available on bioRxiv and Sciety. Each of the reviews posted is the result of comments contributed by 5-9 crowd members. Initial feedback from crowd members suggests that they like the collaborative nature of the reviews: they can reply to each other’s comments for clarification or endorsement, which helps make the review feel like a team effort.
The Sciety Team
The trial will continue until the end of November. We encourage preprint authors who would like feedback on their cell biology papers to contact ASAPbio for inclusion. You can reach them on Twitter (@ASAPbio_) or email to iratxe.puebla@asapbio.org.
ASAPbio will be learning from this experiment by surveying preprint authors and the crowd members to learn more about their experience and will share what they learn from this review activity after the trial is completed.
Sciety is focused on enabling efforts such as this by creating an enhanced technology that increases awareness of and access to their outputs. By joining forces with Groups like ASAPbio who are pushing innovative ways to improve preprint reviews, the process of improving preprint review benefits from the community effort.
To keep up with what this Group is reviewing, login to Sciety, select Groups on the menu, and follow them. Their activities will be added to your own personal feed of events, updated whenever a new evaluation is completed.. For other ways to keep up-to-date with events on Sciety, you can use any of the following media;
Send us an email here
Follow us on Twitter
Like our page on Facebook
