Summary answer:

Séralini used ten rats per sex per group – the same number of animals as Monsanto analyzed for blood and urine chemistry in its 90-day tests claiming to show that GM foods are safe. This is the same number that the Organisation for Economic Cooperation and Development (OECD) recommends for a 90-day subchronic test of the type that Monsanto does on its GM foods, as well as for one of its chronic toxicity protocols. According to statistics experts, groups of this size are enough to show toxicity, but not enough to show safety. This means that industry toxicity studies on this number of rats that claim to show safety are inadequate.

 

Detailed answer:

Critics claim that Séralini used too few animals, meaning no conclusions could be drawn from his results. But Séralini used the same number of animals – ten per sex per group – as Monsanto analyzed for blood and urine chemistry measurements in its 90-day studies on commercialized GM foods, including its study on NK603 maize.1 2 3

Séralini’s critics apparently believe that tests on ten animals per sex per group are sufficient to prove safety, but not enough to prove risk. They are applying double standards. Worse, they have inverted the precautionary principle, which places the burden of proof on industry to prove its products safe. It is not the job of independent scientists and the public to prove to the satisfaction of industry and regulators that a product is dangerous. It is enough to demonstrate there are sound scientific grounds for concern.

Interestingly, the actual number of rats in each group in the Monsanto studies was 20 per sex. Monsanto measured the organ weights of all 20, but only reported the analyses of ten out of 20 animals for blood and urine chemistry.1 2 3 It is not clear how Monsanto chose which rats to analyze: it could have chosen the healthiest rats or those with the healthiest organ weights. This selective practice introduces the potential for bias and invalidates Monsanto’s results.

Some critics say that ten rats per sex per group is acceptable for a short 90-day test, but not enough for a longer chronic toxicity study. But the number of animals used in different experiments vary widely. Much depends on factors such as the purpose of the experiment and the effects (endpoints) being looked for. Another factor is how carefully the researchers have minimized experimental variables, such as ensuring that the animals are the same age and weight. When these are strictly controlled, fewer animals may be needed. This is because any differences seen are likely to be due to the substance being tested.

Even the protocols set by the Organisation for Economic Cooperation and Development (OECD) for industry safety tests differ widely regarding the number of animals required and analyzed:

  • OECD 408, the protocol that is most often adapted by the GM industry for the 90-day rodent feeding trials that Séralini decided to extend to a long-term period in his study, requires ten animals per sex per group.4 This is the same number that Séralini used.5 It is also the same number that Monsanto analyzed for blood and urine chemistry in its 90-day tests on GMOs.1 2 3
  • OECD 453, the combined chronic toxicity and carcinogenicity protocol, requires ten animals per sex per group for the chronic toxicity phase (the same number used by Séralini), but 50 per sex per group for the carcinogenicity phase.6
  • OECD 452, the chronic toxicity protocol, requires 20 animals per sex per group, but only ten per sex per group have to be analyzed for blood and clinical chemistry.7
  • OECD carcinogenicity protocol 4518 and the carcinogenicity phase of 453 require 50 animals per sex per group.

An important point, given the controversy over what conclusions can or cannot be drawn from Séralini’s study, is that OECD 453 cautions that the interpretation of findings in the chronic toxicity phase relies on the larger number of animals (50 per sex per group) in the carcinogenicity phase of the experiment.

Séralini did not have the resources to do a full-scale carcinogenicity study. Accordingly, he did not draw conclusions about carcinogenicity and did not perform a statistical analysis on the tumour incidence or mortality effects. He simply noted details of the tumour occurrence and growth in all groups, in line with rigorous scientific practice and the requirements of the chronic toxicity phase of OECD protocol 453.6

The fact that Séralini’s study was not a carcinogenicity study does not mean that the findings of tumours can be ignored, as Séralini’s critics appear to demand.

Those who make this argument must face its logical consequences. They are in effect recommending that any findings of tumours found in an industry chronic toxicity study following OECD 453, with ten rats per sex per group, should not be followed up and that the product or substance being tested should be allowed to be marketed.

The responsible course of action is to take account of the findings of tumours in the chronic toxicity test and require that a full-scale carcinogenicity study is carried out. Such two-year carcinogenicity studies must be required for all GMOs before they are released into the food and feed supply.

Statistics experts challenge the “too few rats” argument

The argument that Séralini used too few rats to achieve statistical significance in the tumour findings has been challenged by experts in statistics.

Paul Deheuvels, the statistics expert of the French Academy of Sciences, said that Séralini’s results “provided very strong evidence sufficient to establish the existence of some unexpected toxic effects from products previously considered safe”.9

Deheuvels’s view was backed by Prof Peter Saunders, emeritus professor of mathematics at King’s College London and an expert in mathematical biology. Saunders argued that the small size of the groups of animals in Séralini’s study made the results more, not less, convincing. This is because using a smaller number of rats makes it less likely that any effect will be observed. So the fact that Séralini found increased tumour incidence even when using relatively small numbers of rats makes his findings stronger, not weaker, as the lower number of rats decreases the chance that a toxic effect will be seen.10

Saunders said larger groups of animals reduce the probability of a false conclusion of safety (a false negative) when there is actually a carcinogenic effect. This is why the OECD correctly insists on 50 animals per sex per group for carcinogenicity studies: to avoid false conclusions of safety.

Saunders explained:

“If [Séralini’s] experiment had not detected carcinogenicity, that might have been because the groups were too small. As the experiment did detect it, that the groups were small is not an issue.”10

Commenting on the “expert” criticisms of Séralini’s study disseminated by the UK-based Science Media Centre,11 Saunders said:

“The scientists who were asked to supply soundbites for the Science Media Centre were quick to object that Séralini and his group had used the protocol for testing toxicity rather than the one for carcinogenesis. Had they taken a moment to ask themselves why the two protocols are different, they would have realised that in using the toxicity protocol (and remember, that was because it was what the experiment was designed to test) Séralini and his group made it less likely that they would detect carcinogenesis. To criticise a result because the experiment was conducted in a way that was more conservative than required is totally unjustifiable.”10

Saunders and Deheuvels also pointed to the many types of effects found in the treatment groups and how they developed over time. These effects, taken together, indicate a real toxic effect that is unlikely to be due to chance or random variation.9 12

Saunders’s and Deheuvels’s interpretations are even supported by the OECD. In its guideline 116 on how to carry out carcinogenicity and chronic toxicity studies, it states that the purpose of using higher numbers of animals is “in order to increase the sensitivity of the study” – in other words, to avoid false negatives or type II errors, when a real toxic effect is missed.13 In the case of Séralini’s study, there is no problem with the sensitivity of the study: toxic effects from the treatments were seen.

The same message comes from the UK government Department of Health’s Committee on Carcinogenicity (COC), which investigated the proper conduct of carcinogenicity studies in rats. The COC addressed concerns about low survival rates in some strains of rat, which can result in a situation where less than 50% of rats survive to the end of a two-year experiment. This in turn means that an experiment can end up having far fewer rats than were included at the start, leading to low statistical power when trying to interpret the results.

Crucially, however, the COC made clear that low survival rates are an issue in the case of a negative result – a conclusion that the substance being tested is safe and not toxic/carcinogenic. This is in order to protect the public from false claims of safety for a substance.

The COC states:

“For a negative result [conclusion of no effect/safety] from a rat carcinogenicity bioassay to be considered acceptable, survival at 24 months should be 50% or greater in all groups (see OECD, EPA and EC guidelines)… Survival in long-term carcinogenicity bioassays should be compliant with current UK and EC guidelines for the acceptability of a negative result from such studies.”14

The COC does not concern itself with numbers of surviving rats in cases where toxicity is found, because in such cases, small numbers are not a problem.

Notably, the COC considered studies on different strains of rat, including Sprague-Dawley (SD) rats “from various sources”, to investigate whether certain strains or origins of rat had low survival rates, which could jeopardise the validity of the experiment. But it only noted poor survival in connection with SD rats bred by Charles River. It did not mention Harlan SD rats.

Given that the COC’s aim was to improve the reliability of carcinogenicity studies, if there had been a problem with survival rates of SD rats obtained from Harlan, the COC would presumably have pointed this out. It seems reasonable to conclude that the Committee found that Harlan SD rats had acceptable survival rates.

Again, it should be remembered that Séralini’s study was not a carcinogenicity study, and that Séralini’s control rats (the correct number, according to OECD 453 chronic toxicity protocol) had better survivability than the OECD guidelines require. Just 30% of male and 20% of female controls died before the end of the experiment. Only the treated rats suffered unusually high mortality, with up to 50% males and 70% females dying before the end of the experiment in some groups on diets containing the GM maize.

Therefore the OECD’s recommendations for higher numbers of rats in carcinogenicity studies do not apply to Séralini’s study, because:

  1. His study was not a carcinogenicity study and he avoided drawing conclusions on carcinogenicity
  2. He did not reach a conclusion of safety that might have been a “false negative”, so no requirement for higher numbers of rats applies in this case
  3. His control rats had survival rates much greater than 50%, again meaning that OECD’s requirements for higher numbers of rats to avoid false negative conclusions is irrelevant.

Timing is key

A former research analyst and statistics expert with a major government agency, who asked to remain anonymous, argues that Séralini’s study must be taken seriously. The analyst said the findings cannot be dismissed on the basis of claims that the sample groups are too small and that the experiment therefore has poor statistical power.

The analyst said the most important aspect of the study is not the mortality or tumour incidence rates, where critics have focused their attention. Instead, the most important aspect is the timing of all the effects taken together, which stands out in most treatments for both sexes. Treatment groups exposed to NK603 maize and/or Roundup developed tumours and organ damage much earlier than controls.

This argues against the idea that the findings were due to chance and in favour of the idea that they were due to the substances tested.

The analyst said, “I saw the timing differences as key, even if the SD rat was susceptible to tumours, because it suggested to me that the treatment was at least accelerating the tumour progression. I did not see any particular sensitivity over time in the SD rats when used as a control.”

The analyst stated that to reject the entire set of results on the basis of small sample sizes is “not defensible”. The analyst concluded: “Bottom line, something is going on in this study that cannot be – must not be – swept away. I conclude that GMOs must be assessed for safety using the lifetime of the test organism.”15

Logical conclusion to the “too few rats” argument

If Monsanto and its allies wish to argue, based on the relatively small numbers of animals in Séralini’s experiment, that all of the toxic effects were due to chance, then Monsanto is free to fund a carcinogenicity study using a larger number of animals. Such an experiment must be carried out by independent scientists with full transparency. But Monsanto must not have market access for NK603 maize and Roundup in the meantime.

Who should pay to test industry’s products?

Publicly-funded researchers like Séralini must not be expected to meet the huge cost of long-term safety tests on industry’s products, which can amount to millions. Dirk Detken, legal affairs manager at the European Food Safety Authority (EFSA), confirms that it is industry’s responsibility to prove the safety of its products.16 Monsanto should have conducted long-term tests of NK603 and the complete formulation of Roundup (not just glyphosate alone) for carcinogenicity and other ill-effects before applying to market them.

There is a problem with industry testing its own products, however. Scientific reviews comparing studies by industry- and publicly-funded researchers on a variety of risky products, from tobacco17 to pharmaceuticals18 and mobile phones,19 have shown that industry studies are biased in the direction of concluding that the product is safe. This has also been found to be true of studies on GMOs.20 21

So the GM industry must not be allowed to test its own products but must pay a fee into a publicly administered fund, which would commission independent scientists to test the product, giving them a clear mandate to deliver scientifically rigorous results.

To enable independent research to be carried out, industry must end its current practice of not allowing researchers access to the necessary test materials – the GM crop and the non-GM isogenic (genetically the same) variety22 23 24 – and must make them available for such tests.

 

 

References:

1.         Hammond B, Dudek R, Lemen J, Nemeth M. Results of a 13 week safety assurance study with rats fed grain from glyphosate tolerant corn. Food Chem Toxicol. Jun 2004; 42(6): 1003-1014.

2.         Hammond B, Lemen J, Dudek R, et al. Results of a 90-day safety assurance study with rats fed grain from corn rootworm-protected corn. Food Chem Toxicol. Feb 2006; 44(2): 147-160.

3.         Hammond BG, Dudek R, Lemen JK, Nemeth MA. Results of a 90-day safety assurance study with rats fed grain from corn borer-protected corn. Food Chem Toxicol. Jul 2006; 44(7): 1092-1099.

4.         Organisation for Economic Cooperation and Development (OECD). OECD guideline no. 408 for the testing of chemicals: Repeated dose 90-day oral toxicity study in rodents: Adopted 21 September 1998. 1998.

5.         Séralini GE, Clair E, Mesnage R, et al. Long term toxicity of a Roundup herbicide and a Roundup-tolerant genetically modified maize. Food and Chemical Toxicology. November 2012; 50(11): 4221-4231.

6.         Organisation for Economic Cooperation and Development (OECD). OECD guideline no. 453 for the testing of chemicals: Combined chronic toxicity/carcinogenicity: Adopted 7 September 2009. 2009.

7.         Organisation for Economic Cooperation and Development (OECD). OECD guideline no. 452 for the testing of chemicals: Chronic toxicity studies: Adopted 7 September 2009. 2009. http://bit.ly/LxJT1Z

8.         Organisation for Economic Cooperation and Development (OECD). OECD guideline no. 451 for the testing of chemicals: Carcinogenicity studies: Adopted 7 September 2009. 2009.

9.         Deheuvels P. Étude de Séralini sur les OGM: Pourquoi sa méthodologie est statistiquement bonne [Seralini study on GMOs: Why the methodology is statistically sound]. Le Nouvel Observateur. 9 October 2012. http://bit.ly/RtPivG

10.      Saunders P. Excess cancers and deaths with GM feed: The stats stand up. Science in Society. 16 October 2012.

11.      Science Media Centre. Expert reaction to GM maize causing tumours in rats [press release]. 19 September 2012. http://www.sciencemediacentre.org/pages/press_releases/12-09-19_gm_maize_rats_tumours.htm

12.      Deheuvels P. L’étude de Séralini sur les OGM, pomme de discorde à l’Académie des sciences [The Seralini GMO study – A bone of contention at the Academy of Sciences]. Le Nouvel Observateur. 19 October 2012. http://leplus.nouvelobs.com/contribution/661194-l-etude-de-seralini-sur-les-ogm-pomme-de-discorde-a-l-academie-des-sciences.html

13.      Organisation for Economic Cooperation and Development (OECD). Guidance document 116 on the conduct and design of chronic toxicity and carcinogenicity studies, supporting test guidelines 451, 452 and 453: 2nd edition: Environment directorate joint meeting of the chemicals committee and the working party on chemicals, pesticides and biotechnology. 13 April 2012.

14.      Department of Health (UK) Committee on Carcinogenicity of Chemicals in Food CPatE. Longevity in carcinogenicity studies in rats: Analysis of a database prepared by PSD: COC statement COC/00/S3. April 2000. http://www.advisorybodies.doh.gov.uk/coc/longevity.htm

15. GMWatch. Comment on Seralini findings and statistics by former government analyst. 1 October 2012. http://www.gmwatch.org/latest-listing/51-2012/14249

16.      Detken D. Email correspondence to Corporate Europe Observatory, 21 December. 2011.

17.      Barnes DE, Bero LA. Why review articles on the health effects of passive smoking reach different conclusions. JAMA. May 20 1998; 279(19): 1566-1570.

18.      Lexchin J, Bero LA, Djulbegovic B, Clark O. Pharmaceutical industry sponsorship and research outcome and quality: systematic review. British Medical Journal. 2003; 326: 1167.

19.      Huss A, Egger M, Hug K, Huweiler-Müntener K, Röösli M. Source of funding and results of studies of health effects of mobile phone use: Systematic review of experimental studies. Environmental Health Perspectives. January 2007; 115: 1–4.

20.      Diels J, Cunha M, Manaia C, Sabugosa-Madeira B, Silva M. Association of financial or professional conflict of interest to research outcomes on health risks or nutritional assessment studies of genetically modified products. Food Policy. 2011; 36: 197–203.

21.      Domingo JL, Bordonaba JG. A literature review on the safety assessment of genetically modified plants. Environ Int. Feb 4 2011; 37: 734–742.

22.      Lotter D. The genetic engineering of food and the failure of science – Part 1: The development of a flawed enterprise. Int Jrnl of Soc of Agr & Food. 2007; 16(1): 31–49.

23.      Lotter D. The genetic engineering of food and the failure of science – Part 2: Academic capitalism and the loss of scientific integrity. Int Jrnl of Soc of Agr & Food. 2008; 16(1): 50–68.

24.      Scientific American. Do seed companies control GM crop research? 13 August 2009. http://www.scientificamerican.com/article.cfm?id=do-seed-companies-control-gm-crop-research

 

Sources of criticism:

Science writer Declan Butler, writing in Nature journal

http://www.nature.com/news/hyped-gm-maize-study-faces-growing-scrutiny-1.11566

European Food Safety Authority

http://www.efsa.europa.eu/en/efsajournal/pub/2910.htm

http://www.efsa.europa.eu/en/efsajournal/pub/2986.htm