Abstract: As part of an $816 million initiative to manage risks represented by possibly hazardous substances, Canada was the first country in the world to determine that the common chemical bisphenol A (BPA) should be classed as “toxic” and accordingly banned polycarbonate baby bottles. The process set up to conduct this risk assessment differed from the previous Canadian experience in that it was more formal, systematic and more pluralistic with much greater participation from interest groups. This case study examines the forces that impacted the regulatory process of BPA and argues that long-term, institutional and legislative forces interacted with short-term interest group politics and public opinion. It argues that the federal government issued a decision that went beyond what was scientifically validated but that reflects a widespread social perception of risk posed by chemicals that was embedded in the legislation governing the Chemicals Management Plan (CMP), public opinion and the media coverage of the issue. It uses existing literature on the nature of risk perception to assess critically the values underlying the CMP and those expressed in the regulation of BPA.
The other day I was rock climbing and someone dropped their glass water bottle, sending thousands of tiny, sharp shards of glass all over the floor, where dozens of people, some young children, were walking around in bare feet. Six years ago, this never would have happened because most rock climbers would have been using hard, reliable, plastic water bottles that were hardened with a chemical called bisphenol A (BPA). Thanks to an ongoing campaign by environmentalists and some scientists, BPA has become a modern day equivalent of DDT. Because of public pressure, retailers of products made with BPA, including baby bottles and outdoor bottles, withdrew their products and replaced them with a wide variety of bottles made from different products, including glass bottles, which, as noted above, have a tendency to break. In essence, people were convinced to act on one risk (the risk supposedly posed by exposure to BPA) and unwittingly opened themselves up to other risks (broken glass). But in all the discussion about the supposed risk presented by BPA, the issue was never framed this way. Sadly, discussions about threats to welfare (risks) usually are not.
This is one conclusion that emerged from my paper published online recently in the Canadian Journal of Political Science that examines the politics and science of Canada’s regulation of BPA. Canada was the first country in the world to regulate it, announcing in April 2008 that it was “toxic” according to the Canadian Environmental Protection Act. The paper argues that this decision was not supported by scientific evidence. In fact, it produces documentary evidence that scientific experts in Health Canada felt that “at this stage, any risk from BPA is hypothetical”. Their own risk assessment described the evidence for the existence of some threat to human welfare as “limited” (see p. 71). Instead, the decision was a product of a widespread suspicion of chemicals that is the product of both cognitive and cultural forces. Continue reading →
In addition, it was the product of very strong lobbying by Environmental Defence and other environmental groups and a great deal of questionable reporting by the Globe and Mail’s Martin Mittelstaedt which emphasized what flimsy evidence there is that suggests there is some threat to health, ignoring the much more robust evidence that indicates the opposite. But what Mittelstaedt and ED ignored, and what Health Canada knew but downplayed, was that all of the evidence that had been produced up to that point (and to this day) was based on flimsy methodologies or showed effects manifesting themselves at levels of exposure far higher than what Canadians are exposed to.
This is an important case for several reasons. First, the fear and suspicion of chemicals is widespread. The Sudbury father who recently sought a vaccine exemption for his daughter is on the record saying: “I don’t believe chemicals should be dumped into our system.” In research I’m doing on the politics of municipal water fluoridation, one of the common charges opponents make is that it is not fluoride that is added to the water, but rather hydrofluorosilicic acid. This compound dissolves into fluoride, but fluoridation opponents don’t know this or don’t care. By focussing on a term that caters to chemophobia, anti-fluoridation opponents can actually overturn fluoridation, an important public health initiative that can effectively and equitably improve dental health for a wide segment of the Canadian population. By acting on flimsy evidence, the federal government legitimates excessive fears of chemicals.
Second, journalists play a key role in amplifying risks. Thanks to Google Trends data, I was able to correlate the frequency of news stories about BPA in Canada with internet search interest about the same topic over several years. You can see the results here.
Correlating newspaper coverage with public interest in BPA.
In nearly every week where there was a spike in newspaper interest in BPA, there was a corresponding spike in public interest in BPA. I’m pretty confident in saying that newspaper coverage (particularly Martin Mittelstaedt’s) coverage sent a lot of worried and curious Canadians to the internet to find out more, making the issue more salient in public opinion. A good example of Mittelstaedt’s reporting can be seen here where he describes BPA as “inherently toxic”. While this certainly sounds frightening, the fact is that BPA was only ever found “inherently toxic” to aquatic organisms, not for humans. Moreover — and my paper spells this out — this criteria was not enough to trigger a full screening assessment alone; at the early stage in the regulatory process, this finding was irrelevant. But Mittelstaedt and others made no mention of this because they didn’t want that fact to get in the way of a good scare story.
And lastly, this case shows the need for a better discourse about risks in politics and public policy. One thing that needs to be better understood is that invoking the existence of some threat to welfare (a risk) is only ever a partial equation. Other elements of that equation include what the quality of the evidence is that establishes existence of that risk. In the case of BPA, it was very poor. Yet another part of that equation asks whether public welfare is actually improved by doing anything about it and if so, what that should be. In this case, some environmental groups like Environmental Defence, Martin Mittelstaedt and Health Canada have valiantly protected us from risks based on some pretty flimsy evidence. And in doing so, they’ve helped take hard, reliable, unbreakable plastic bottles off the market place. And now, rock climbers, outdoor activists and parents are using glass baby bottles protected from a hypothetical risk, and now exposed to the risks posed by broken glass.
Clearly this is not the most tragic case of misperception of risks. But in other domains – such as how we try to deal with supposed threats from terrorists or try to minimize the risks from pesticides – failing to appreciate how dealing with one risk can expose us to others could make us all much worse off.
There’s been a lot of commentary on the consequences of the Rogoff-Reinhart error, which is seriously undermining the empirical basis for the austerity program that has swept the US, Canada and Europe. I think this episode highlights the limits of evidence-based policy-making.
One clue that there is deep limits to the concept is that I’ve never actually heard a policy-maker claim that they are making policy without evidence. It seems that everyone justifies their policy preferences with some sort of evidence; it’s just that there is always a debate about what constitutes evidence. I also think that many liberal (and here I mean, small-l liberals with a strong commitment to equating justice with procedure, rather than outcome) scholars and citizens adhere to an ideal of evidence-based policymaking. I think it’s because they feel that somehow the nastiness of politics can be minimized in public policy making if we can all just agree on the facts and procedures for adjudicating facts.
Here I’ll make two claims: evidence can only inform, it can never resolve, political debate. And evidence-based policy-making will always be influenced, if not, determined by pre-existing values, rather than the other way around.
If evidence really could influence policy to the degree that the adherents of evidence-based policy-making claim it can, then we would be seeing a much quicker reversal of economic policy given that one of the essential pieces of evidence for austerity has been shown to be, well, wrong. Aside from this in Italy, austerity remains the dominant economic policy. Continue reading →
It’s true it is early days and this development may ultimately shift global economic policy, but, the longer it takes, the more it proves my point. Evidence cannot be decisive for policy-making; instead, it interacts with deeper values which are of greater import. In terms of austerity, it wasn’t R-R that told policy-makers to adopt austerity; I assert that it fit with widespread, deeply-held values that a good life is one that is lived modestly with an emphasis on earning and saving money.
I write this post only partially motivated by the fall-out from R+R. But this episode reinforces some of the conclusions that I’ve drawn out of my ongoing research into the politics of the regulation of BPA in Canada and the US. Clearly, evidence mattered in that case. Both sides – those in favour of and those opposed to stricter regulations – marshalled as much scientific evidence as possible to support their claims. And both sides are guilty of calling the other “anti-scientific”. The problem is that there is no agreement on what constitutes evidence. Those who assert there is a great threat posed by exposure to BPA often rely on studies which often do show adverse effects from exposure to BPA, but base this conclusion on studies that only administered one or two different doses of BPA. When these small studies show a positive relationship between exposure and effect, environmental activists seize on them because they have already made value commitments that we are constantly being poisoned by toxins and that modern society is killing us.
It has to be emphasized: there are powerful structural forces that govern the scientific endeavour that leads to the publication of these kinds of exciting findings based on small sample sizes and dose ranges. Put yourself in the shoes of a new, untenured, natural scientist. You need to get grant money and you need to get publications. In tough economic times and the competitive scientific job market these pressures can be excruciating. But null findings just won’t cut it. Journals that are owned by a small oligopoly of media companies want citations to papers that they publish and tend not to attract a lot of notice. Findings that show that a substance does have an effect is likely to get cited.
The vast majority of positive findings are published when they show effects that are “statistically significant at the .05 level”. This means that there is only a 5 per cent chance that that published finding will have occurred by chance alone. But what that means is that even if scientists are entirely objective, then 1 in 20 scientific studies will be a false positive. However, scientists are not entirely objective. Too often, null results from studies are shelved and only the statistically significant studies are published. But, if we are never told how many null findings the scientist received before getting a statistically significant finding, then we have no way of determining whether or not the finding could be random noise or a real finding. Ben Goldacre has pointed out that this badly plagues pharmaceutical research, but Ioannides has found similar problems in medical research.
There are two wonderful images from a presentation to an expert working group on BPA from June 2012 that prove this point better than I ever could. This is around 50 random values ranging from 80 to about 145 plotted on the y-axis.
Would you say that there is any pattern evident? Of course not.
Now, would you say the same about the following four values?
That looks to me like BPA/coffee/smoking/cosmetics causes cancer/obesity/heart attacks!
Except the values are drawn from the same randomly selected 20 values in the first figure. Look closely, I can see at least two sequences of 4 randomly generated values that increase monotonically creating the impression of a linear, cause-and-effect relationship.
This is what gets published, even if it’s the product of random noise.
There are ways to guard against this, but they essentially involve doing large-scale, multi-generation rodent studies with a wide range of doses. But this costs a lot of money and sadly,the only people who have this kind of money to do research (which often results in null findings) are corporations who obviously have a strong interest in the publication of such findings when substances they produce are under scrutiny.
Just because it is funded by corporations does not mean the “evidence” is invalid according to strict scientific standards, but we’re not dealing in science: we’re dealing in politics. And this fact certainly excites activist groups, politicians, journalists, rival scientists and voters who see these kinds of studies not as “evidence” of no findings, but of “evidence” of collusion and corruption.
Into this vortex of institutional, economic and value-based judgements, politicians and regulators are called on to make a decision. They have developed some tools to improve the role of “evidence” in policy making, primarily through the use of systematic literature reviews and tools to evaluate the merits of different type of evidence.
But as I stated before, this always interacts with pre-existing values. An excellent example of this in the Canadian Environmental Protection Act. There, the Ministers of Health and the Environment are required to apply both the precautionary principle and the weight-of-evidence approach when publishing risk assessments of potentially toxic substances. The former, in one form, requires (some might say allows) regulatory action even based on inconclusive “evidence”. The latter requires regulators to rationally weight the merits of “evidence”, discounting weak “evidence” such as studies with small dose ranges, and according greater weight to large-scale, more credible, more reliable studies.
With the requirement to apply both principles entrenched in Canadian legislation, you can see how “evidence” can never answer the regulator’s question of what type of policy she should adopt. There is always a debate about what constitutes “evidence”. Those who are convinced that inconclusive “evidence” is actually sufficient “evidence” for action, will simply demand that the state act, those who disagree that the “evidence” is conclusive will simply state so. There actually is no “evidence” that can reconcile these two positions, because this is actually never about “evidence”; politics is about values and interests.
That’s why the R+R findings made such a splash in the first place, why their refutation won’t matter much, why BPA was deemed “toxic” even though the scrreening assessment explicitly states that it is safe at current levels of exposure and, frankly, why evidence-based policy-making is not so much impossible, just illusory. It can’t exist as the kind of decisive resolution to political challenges people want it to be.
There is news today that a coroner in New Zealand formally ruled that the excessive consumption of Coca Cola led to the untimely death of Natasha Harris three years ago. Harris, it seems, developed a nasty habit of drinking up to 10 litres a day. “I find that when all the available evidence is considered, were it not for the consumption of very large quantities of Coke by Natasha Harris, it is unlikely that she would have died when she died and how she died” is what the coroner determined.
So what does this have to do with a blog about Canadian, politics and public opinion? Because this tragic incident illustrates a bedrock principle of toxicology that citizens, journalists and politicians often forget when our attention turns to issues of concern with possibly toxic substances. That principle is “the dose makes the poison.” It was formulated in the 16th century by the Swiss-German chemist Paracelsus and it holds that all substances (Coca-Cola included) are toxic in some quantity. Even something as seemingly innocuous as water can be toxic when enough has been consumed. This poor soul died from the excessive consumption of water during a brutal hazing ritual in New York. From this observation, the traditional role of toxicology has developed which is to ascertain at what level of exposure any given substance causes adverse effects.
With the rise of concerns about toxic substances in the post-World War II era, scientists have discovered some substances that deviate from this principle (see tamoxifen, an important drug in chemotherapy). But these effects are rare, difficult to demonstrate and the scientific debates that surround them are heated and complex. But when issues shift into the public and media spheres, the complex science behind these assertions, sadly, usually gets lost in the shuffle. Continue reading →
There are multiple reasons for why this is. One reason is our own inability to perceive risks rationally. There is so much literature on the psychology of human risk perception that it is difficult to know where to start, but I might recommend this thispaper or this book. The long and short of the field of risk perception is that humans get very frightful about lots of things that actually present very little danger. After September 11th, Americans turned away from planes for their long trips and hopped into their cars, leading to a sharp increase in traffic fatalities in the months following.
Chemicals are one of those things that people fear a great deal. Ask people if they think the colourless, odourless chemical dihydrogen monoxide (the ingestion of which is the cause of thousands of deaths each year) should be banned and you’d be surprised (and a little concerned) at the number of people who would agree.
So when the public and the mass media turn their attention to issues of possible harm from exposure to chemicals, fears can be stoked, perspective can be lost and can lead, ultimately, to over- and unnecessary regulation of products and activity. For example, I was able to show in this paper that within-state newspaper coverage of Bisphenol A was positively linked to the chance that a US state legislature would introduce or adopt legislation banning products made with BPA the following year. It showed (again) the important role that increased media salience can have on policy changes. This, even though, scientific information suggests that exposure to BPA is hundres, if not thousands of times lower than the level at which there is a consensus that it causes adverse effects in rats (see here).
This research area is rife for investigation and it’s become a long-term project I’m working on, so I’m going to have a lot more to say about this in the future. I thought the news about Ms. Harris’ tragic death due to the excessive consumption of Coca-Cola was a useful way to set the stage for more thoughts and research to come.