Developing a teaching resource on peer review
Submitted by sis on 06 June 2007
Changes in the English national science curriculum will allow more time than ever before for exploring the inner workings of the scientific method (see Burden, 2007). How scientific ideas are presented, evaluated and disseminated is now a core requirement for key stage 4 students (ages 14-16). At Sense About Science, a charity to promote good science and evidence in public debates, we welcome this emphasis. Students need to discover early in their education what science is, if they are to handle data and evidence maturely and with discrimination, and gain their first insights into mastering a subject rather than being mastered by it.
This is important not just for understanding the scientific method but also for helping students to negotiate the world outside school. We need students to understand that the scientific knowledge we now regard as established fact – such as the Earth revolving around the Sun – is actually the result of many years of academic argument and gathering of evidence. In this way, students can be encouraged to consider new research critically, to consider its evidence base, and not simply to believe new theories because they appear to make sense. At the moment, there are few resources available for teachers to work out how scientific information is evaluated and added to the ongoing body of knowledge.
In 2005, Sense About Science published I don’t know what to believe…,a short guide to peer revieww1. It has been hugely popular with over 60,000 copies disseminated worldwide. The guide aims to help people understand the journey that scientific ideas undertake before they tentatively join the body of scientific knowledge and other researchers can begin to repeat them. The guide also highlights the difference between published and unpublished research, helping people to determine whether the research claims they read in newspapers or on the Internet come from scientific journals or from scientists who have chosen not to subject their work to the critical scrutiny of their peers.
The first time that most students learn about the system is when they study a science subject at university, which means that those who do not continue with science will probably never know the role that peer review plays in the advancement of science. Following the guide’s release, we received many requests for its use in the classroom. These requests came from primary- and secondary-school teachers, and others involved in science education.
We have always been keen to develop a resource specifically for schools on peer review and the acceptance of new scientific ideas. It was decided, therefore, that rather than providing schools only with I don’t know what to believe…, we would begin to build a full education resource, with a web resource centre, where teachers and students alike could access further information about peer review, scientific knowledge and evidence. The resource is now under development with a planned completion date in summer 2007.
In addition to the web resource, we will also produce free printed materials for teachers. The printed resource will include the nuts and bolts of the peer-review process and ‘day in the life’ stories from journal editors and scientists. There will be case studies of media controversies over scientific research, such as the measles-mumps-rubella (MMR) vaccine and genetically modified foods, to look at how scientific evidence is interpreted by different stakeholders. Finally, there will be practical exercises where students will be asked to critically evaluate one another’s work. The resource, like the short guide, will not shy away from difficult questions, such as how to deal with fraudulent results or maverick scientists who head straight to the media with their results. Confronting difficult questions will, we hope, lead to stimulating classroom discussions.
Although we must fight to retain the practical aspects of science education and the teaching of a broad scientific knowledge base, discussion about the applications of science and the ethical and societal implications should not be seen as detrimental or as the ‘easy’ option. Providing future scientists and consumers with a true understanding of the scientific method, its theory and practice, and how ideas and evidence are generated, should produce individuals who are well able to hold their own and stand up for science, particularly in controversial areas where it is not easy to see the path ahead.
How to get involved
Sense About Science is recruiting science teachers to trial the resource. We want to ensure that it hits the mark in the same way as I don’t know what to believe… has. If you would like to be involved, we will email you the resource with questions to raise with your students. Although the resource is based on the English national curriculum, teachers across Europe are invited to get involved in the trial – peer review is, after all, an international process!
The resource will be freely available online when finished. If you are interested in being part of the review team, or in finding out more, contact Ellen Raphael: firstname.lastname@example.org
What is peer review?
Peer review ensures that a research paper has been checked by other qualified scientists for mistakes and omissions, as well as to clarify what the findings show. It also means that the results are available to the wider scientific community, so that others in the field can try to replicate the findings, or use them, in conjunction with other work or results, to reach further conclusions.
Scientific research that has not been subjected to this form of review is of no help to anyone. Scientists cannot repeat or use it and society cannot base decisions about public safety – or individual health, for example – on work that has a high chance of being flawed. The need to clarify the status of scientific evidence is growing. In the UK, the government increasingly uses public consultations, inquiries and commissions to gather evidence for regulatory purposes and everyone involved needs to be clear about the measures of scientific plausibility.
So, no matter how exciting or compelling new scientific or medical research is, you must always ask, “Is it peer-reviewed? If not, why not?”
If it is peer-reviewed, you can look for more information on what other scientists say about it, the size and approach of the study, and whether it is part of a body of evidence pointing towards the same conclusions.
Edited extract from I don’t know what to believe…w1
Case study: mobile phones
One of the first claims that mobile-phone emissions are unsafe was made by Roger Coghill, a self-employed researcher who had previously argued that mobile phones cause headaches and memory loss. In 1998, Coghill said that the waves produced by mobile phones could damage the activity of lymphocytes in the body’s immune system. Coghill published these claims himself and released them to the media, rather than submitting them for peer review. Many other studies failed to show damage to the body’s immune system as a result of mobile-phone usage. Despite the lack of corroboration, Coghill’s claims were widely reported, and fuelled discussion about mobile-phone safety. Between 1998 and 2003, he was cited in 119 printed news publications in the UK, most of which made no reference to the lack of peer review of his research or to the fact that other, peer-reviewed research did not corroborate his hypothesis.
More recent large-scale peer-reviewed studies have also found no evidence that mobile phones cause harm, and Coghill’s hypothesis still remains unsubstantiated. Claims like Coghill’s, which have not been scrutinised by scientific experts through the system of peer review, cannot be validated, and therefore, no matter how widely they are reported, such claims remain, essentially, just an opinion.
Burden J (2007) Twenty First Century Science: developing a new science curriculum. Science in School 5: 74-77.
w1 - I don’t know what to believe… is freely available from the Sense About Science website.
Hard copies can also can also be requested from this page. There is a small charge for sending print copies outside the UK, to cover postal costs.
For other useful education materials about peer review, see:
Gift N, Krasny M (2003) The great fossil fiasco: teaching about peer review. The American Biology Teacher 65: 270–278. DOI: 10.1662/0002-7685(2003)065[0270:TGFFTA]2.0.CO;2
Even if the topic is not as exciting as the most recent discoveries in cutting-edge research, this article is relevant for secondary-school teachers and students in science education. In fact, the focus on peer review, in addition to the usual sequence of ‘observation, hypothesis, experiment, theory’, is necessary for a correct epistemological approach to what makes science science.
For these reasons, Ellen Raphael’s clear and concise article is suitable for teachers interested in updating their knowledge and for students motivated to deepen their study of the scientific method. Moreover, the invitation from Sense About Science to test the didactical materials (printed guide and related website) is also a good opportunity for teachers interested in the societal impact and perception of science. The full article could be used for classroom activities or read by students interested in the topic.