Yesterday, John Roughan’s column in the Herald was about the people of Christchurch and their need to know “the worst they might face”. He draws his story from the reluctance of the GNS to make comments about the possibility of further strong earthquakes even though they knew there was good scientific evidence that there would be further large shocks. While I have plenty of sympathy for the people of Christchurch and much admiration for the way they have stood up to things so far, I do not think that Roughan is correct in his desire to see such scientific information released. I do not think it would be helpful to the long-suffering people of Canterbury.
The problem is that Roughan, like most lay people, does not understand the wide range of levels of evidence available to scientists. In our day-to-day lives, we mostly encounter two types of evidence – established (or empirical) fact and testimonials. We either know something is true or we think it is true because someone (whose opinion/knowledge we respect) told us so. Science uses these types of evidence alongside a third – statistical evidence. There are multiple levels of statistical evidence.
Obviously, experimentally verifiable fact is the highest level of evidence available to science. Unfortunately, many lay-people think that when they hear a scientific expert speak, s/he is speaking from this level of evidence. This is rarely so. Most are talking from some level of statistical evidence. This is because we are usually dealing with very complex systems that are difficult to experiment on.
If we are lucky, we can be making our statements based on level A evidence. This is the level of the randomised, double-blinded clinical trial in medicine. Readers of MacDoctor will know that this level of evidence is by no means the same thing as established fact, but it is good evidence, nevertheless. Sadly, this type of controlled experiment can only be easily performed on closed complex systems such as people and animals. Geological systems are much harder to perform experiments upon.
The commonest level of evidence available in most disciplines is Level B evidence. This would be the equivalent of observational and cohort studies in medicine. The system is too large to easily manipulate and it can only be observed and studied. Readers of MacDoctor will also know that this sort of evidence is very prone to confounding effects where it becomes very difficult to prove that correlation is actually causation. In medicine, conclusions made from this sort of data are then tested with Level A studies, often leading to the opposite conclusion. This option is not available to geologists, climatologists, vulcanologists and other disciplines dealing with large, open-ended stochastic systems. These disciplines rely on accumulation of large amounts of Level B evidence from multiple sources to refine their theories.
The “lowest” level of evidence is again one that a lay-person would be more familiar with, the level of experience and expert opinion. While this type of evidence is indeed extremely useful, it is not the type of evidence that we should be formulating major policies upon. This is often really the only available information for unique disaster situations – not necessarily a bad thing in the initial stages of a disaster, but a real problem when it comes to long-term planning.
The people at GNS would not have concerned themselves much with the public’s right to know. After all, they are scientists and the scientist’s raison d’être is to impart knowledge. No, the question they would have wrestled with is “does the level of certainty of this information warrant the possible consequences of releasing it?“. Like John Roughan, most people will think this information is like any other “fact” that they know and accept it as “truth”. Unfortunately, a 90% certainty of another magnitude 6 quake in the next 6 months does not tell us a lot if the quake’s epicentre is 100Km deep and 50 Km out to sea.
It is exactly this sort of uncertainty that makes it so difficult for experts to speak out; especially experts with good credibility whose word will therefore be easily accepted by the public as gospel. What if the revealing of this information had many thousands more leaving the city of Christchurch, leading to a complete collapse of Christchurch’s economic structure? What if it then turns out that the quakes either did not occur or were trivial? This is a piece of information where the consequences are very different to a false tsunami warning, which is merely an inconvenience.
Alternatively, the geologists could try and couch the information in dozens of caveats. This will sound to the general public like the experts don’t really know anything or that they are dismissing the quakes as unlikely (people hear what they want to hear unless the message is emphatic). What if there was then a quake in which there was loss of life? Who would then be “to blame”?
Roughan would like the people of Christchurch to cease being in the limbo of uncertainty. The GNS would like that too. Unfortunately, despite good, educated opinion and evidence, they cannot provide the sort of certainty that Roughan is hoping for. Nature may be susceptible to analysis on a wide scale but is wildly unpredictable in any one locality.
That is the only real fact.