This is article is currently a Featured Commentary at the Melisa Institute.
To view the original please visit the chileno blog
Are abortion rates overestimated?
An article published today in the journal Ginecología y Obstetricia de México, serves as the latest chapter in a long-standing debate over the measurement of abortion rates and maternal mortality. Earlier in 2012, a research group lead by Chilean epidemiologist Elard Koch showed that the maternal mortality ratio (MMR) had declined some 93.8%, from 270.7 to 16.9 maternal deaths per 100,000 live births in the period from 1957-2009. The authors argued that progress was being made in areas of maternal health and that there were reasons to be particularly optimistic in the case of Chile as the country displayed one of the lowest MMRs in the American continent, second only to Canada. In what proved to be somewhat controversial, the authors also claimed that this long-term progress was largely unaffected by the introduction of legislation prohibiting abortion in 1989. That is, in this natural experiment, maternal health had continued to improve over time regardless of the legal status of abortion in the country.
Maternal health, mortality and abortion are all understandably thorny issues which naturally elevate the passions and emotion can run high in debate and discussion, as well as in the scientific literature for a host of reasons ranging from the humanist, religious to the ideological. In sharp contrast to the view put forward by Koch and colleagues, researchers predominantly associated with the Guttmacher Institute have argued that abortion rates have ceased their historical fall, are stagnating and that abortion in countries with legal restrictions leads to more abortion, not less, and also to an increase in potentially fatal unsafe abortions. Arguably, the diverse findings reported in the literature are related to the difficulties that arise when trying to measure the phenomenon of interest, which is fraught with problems; Women tend to underreport abortions, hospitals, perhaps particularly private clinics, may not record them properly or even attempt obfuscation. Clandestine abortion is very difficult to estimate. This situation is exacerbated by regional variation in cultural values and norms as well as laws in place that, to varying degrees, prohibit abortion. To try and understand further the problems of measuring sensitive events like abortion and related complications, what follows is a review of the current debate which is also the focus of the article published today in Ginecología y Obstetricia de México. It is focussed on the issues surrounding data analysis and methodological procedures. We received views from Dr Koch from the Institute of Molecular Epidemiology of the Universidad Católica de la Santísima Concepción, from co-author Dr Calhoun a specialist in Obstetrics and Gynecology from the West Virginia University-Charleston and Dr Sedgh from The Guttmacher Institute.
The UN Millenium Development Goal 5 (MDG5) on maternal health has among its aims a three-quarter worldwide reduction of the MMR by 2015. The primary causes of which for developing regions are shown in Figure 1. As can be seen, more than 50% of cases are likely explained by haemorrhage and hypertension while a further 29% are the result of direct (e.g. ectopic pregnancy or complications with the caesarean section) and indirect causes (e.g. malaria or heart disease).
Figure 1. Source: MDG Report 2010. Indirect causes include heart disease, malaria and HIV/AIDS while other direct causes refers to obstructed labour, complication at caesarean section or anaesthesia and ectopic pregnancy.
Nevertheless a further 9% is attributable to abortion and miscarriage, a factor which has been the subject of much debate and controversy over the years. In many quarters it is believed that progress has been made on the MDG5 goal of reducing MMR in most parts of the world including Latin America. World Health Organization (WHO) figures report that overall maternal death rates due to unsafe abortion have reduced in all corners of the globe and to the tune of 60% in Latin America (see Figure 2). However, as alluded to above, a recent article published in the prestigious medical journal The Lancet, suggests that although abortion rates had been falling from a high of 35 per 1000 in 1995 this trend had entered a stable period of 28 to 29 in 2003-2008 and that unsafe abortions were actually on the rise. In addition the authors suggest that countries with restrictive laws did not have lower abortion rates, rather the extent of liberal abortion law was negatively* related to abortion rates. On the latter point, it is instructive, as always, to take a closer look at the data. In particular it is important to note that the data were not modelled in terms of proportional differences, but by correlating the proportion of women living under liberal abortion laws with the abortion rate per se. Although the association reached the minimum level of statistical significance usually accepted in the scientific literature (remember that significance correlates highly with sample size; r+(pn), the magnitude of the relationship was in reality rather modest. A beta from a linear regression model was reported of (sic) β = 0.1. Given that β is standardised and equates to a correlation coefficient, we can estimate that the variance shared between these two factors is approximately 0.01%, given of course a dose of measurement error. But it is an intriguing if somewhat counter-intuitive finding. Clearly further research is needed to better understand its practical significance. It is certainly counter to Koch's work who consistently suggests that where more liberal laws are in operation, abortion rates increase, another source of debate.
The second point of the article was an important one concerning unsafe abortion, and the data provide prima facie evidence of a worrying upward trend of potentially fatal unsafe and clandestine abortion practices. However, this conclusion has also courted some controversy. It should be pointed out that while 'unsafe' abortion defined as, "a procedure for termination of an unintended pregnancy done either by people lacking the necessary skills or in an environment that does not conform to minimum medical standards, or both” by the World Health Organization (WHO) is thought to be responsible for the death of thousands of women worldwide, paradoxically, abortion carried out under proper supervision is 14 times safer than childbirth. Similarly, when compared with more clandestine methods such as inserting objects into the uterus, the use of caustic agents such as misoprostol are relatively safe. This is an important point to bear in mind as the definition and categorisation of abortion and mortality, as well as live birth counts are all key methodological factors that determine the conclusions drawn from empirical investigation and ultimately inform national and international health policy. This study did include clandestine medical abortions as well as those outside a medical setting. However, as Beverly Winikoff, writing in The Lancet has opined, many medical abortions in the US are now carried out in private homes and yet are not considered unsafe. She argues that the WHO definition is outdated and suggests a re-evaluation of what constitutes safe and unsafe abortion. Although the rate of abortion, per se was not disputed by Winikioff, the volume of unsafe abortions was. It is entirely plausible, therefore, that 'unsafe' abortion rate estimates could be inflated if the definition is so wide as to encompass abortion procedures that although may deviate from normal protocols, may not actually carry a particularly high risk of insult. In other words the rate of abortions deemed unsafe could in fact be much lower than is currently thought. Dr Sedgh comments that, "Dr. Winikoff is referring to the classification of abortions as safe or unsafe. She suggests that some of the abortions that classified as “unsafe” are actually fairly safe, partly because the growing use of misoprostol in some countries has allowed some previously unsafe abortions to now be less risky or dangerous to the woman, even though they are still clandestine and against the law." The good news is that this is a testable hypothesis that can be addressed in future research.
Figure 2. Source: WHO Unsafe abortion incidence and mortality
The primary finding of The Lancet study of no or little progress in maternal health in recent years, is profoundly important not only in terms of meeting the UN MDGs but also more generally for individual nations in informing their decision-making on policy, especially in these austere times one might argue. The veracity of the claims of little progress is crucial. A debate has arisen over whether it is in fact possible that true progress has been achieved, yet is somehow clouded to some extent by difficulties with measurement. For a host of reasons data available on abortion can be patchy, unreliable and invalid in many countries, particularly those in developing regions and countries where the practice is prohibited by law. Winikoff speaks to the problem in her critique of The Lancet paper noting that, "Even in countries with liberal abortion laws, estimation of abortion incidence is problematic and relies on the use of many data sources and applications of adjustment factors. The challenges are even greater in restrictive settings where the use of indirect methods of measurement is required." Such adjustment and correction factors are rife in some of the published papers in this area including that published in The Lancet this year. Arguably the most problematic methodology is the indirect assessment of abortion figures. It has to be acknowledged that in circumstances where data are sparse or unavailable, and this is most difficult in countries where abortion is illegal, it is a necessity to find alternative methods to medical notes. Indirect assessment has taken the form of asking clinic staff, albeit presumably well informed, to retrospectively estimate the average number of abortions seen in clinic. Koch and colleagues argue that many staff members typically asked to retrospectively estimate typical numbers of abortion are not likely to be able to provide accurate estimates, while the Guttmacher Institute researchers strongly disagree. The danger here is that a range of known psychological processes could be operating in the survey including recall bias, social desirability, realising the 'demand characteristics' of the study or deception. Yet at the same time it is possible that a reliance on estimation and indirect methods may not be fatal to research. It is known that rates of abortion obtained from questionnaires are under-reported, for reasons of stigma, fear of social and legal repercussions and so if anything the numerator in rate calculation could indeed be high. Dr Koch concedes that, "when a sample of women of reproductive age is directly surveyed about induced abortion over a specific lapse of time, abortion generally will be underestimated. However", he continues, "it is very difficult or virtually impossible to establish figures on abortion using opinion surveys applied over a limited number of informants such as health workers." And it is this criticism that has become a central tenet of the Koch critique on alternative estimation methods and deserves closer inspection.
In a paper published in the International Journal of Women's Health Koch et al. outline this thesis in the case of Mexico, challenging the use of opinion surveys in abortion research that attempt to indirectly obtain estimates by asking health care professionals to give the number of abortions seen in a 'typical month' and in 'the current month'. These two estimates are averaged and multiplied by 12 to give an overall best guess as to the number of abortions carried out in a given clinic over a year-long period. The study found that, in a natural experiment in Mexico where abortion was legalised in the Federal District of Mexico in 2007, for the period 2007-2012 (a period thought to be long enough for legal abortion to replace illegal), when compared to objective clinical notes, the opinion survey estimates were some 1000% overestimated. Furthermore the authors calculate that for the cumulative five year period, the total number of legally induced abortions (78, 544) was approximately 50% the number given for a single year in the original study under scrutiny. Dr Calhoun reasserts that these, "methodological inconsistencies to estimate abortion statistics may lead to erroneous conclusions about the progress of maternal health in Latin America. This issue has plagued the abortion discussion for decades." It is not clear whether abortion rates are going to represent a constant in clinical settings or whether there will be seasonal variation. The difficulty with accurately estimating prior events, even personal ones has a long history in research.
One way to assess whether an indirect method is indeed valid is to compare it against a gold standard measure. On the use of the measure in validation studies, Dr Sedgh commented that, "The study is employed in countries with incomplete medical records; it would be a waste of resources to conduct this study where abortion statistics are complete. However, aspects of our work and our overall estimates have been corroborated." On this point I strongly disagree, it is standard practice in psychometric testing to provide evidence of validity in the form of concurrent, divergent and convergent validity as well as test retest reliability and inter-rater reliability. That is, does the measure tap into the information that you think it does, and do you get the same result when repeating the assessment or when using the assessment with different raters or different groups of people. Rather than a waste of time, such validation studies are routine in psychological science. Instruments are rarely taken seriously without such psychometric validation and in my view rightly so.
The second main criticism of studies classifying abortion related deaths is in the International Classification for Diseases (ICD) codes used. Koch and colleagues point out that including deaths related to ectopic pregnancy, for example, has little to do with the target of interest (i.e. induced abortion mortality) in the same way that hepatitis in the liver would not tell you anything about alcohol derived liver disease. Dr Calhoun comments that, “For instance, the nine codes related with death with abortive outcome are often grouped as if they were all associated to illegal induced abortion; this is clearly inappropriate since ectopic pregnancy, spontaneous abortion, abnormal products of conception and medical abortion are unrelated to illegal abortion”. In addition the Mexico study claimed that 98% of maternal deaths were not induced-abortion related. For the most part causes included haemorrhage, hypertension and eclampsia and other pathological conditions. This finding chimes with the WHO report illustrated in Figure 1 and makes it difficult to see how the legalisation of abortion can make a major impact on MDG5 MMR reduction. That is not to say that induced abortion is not an important issue of course, just that in the grand scheme of things it appears to make a relatively contribution to maternal mortality. But as the saying goes it gets tiresome always being in the minority group.
Back to the paper appearing today in Ginecología y Obstetricia de México, Dr Koch and colleagues have provided a rebuttal to a response by Singh et al. in August of this year to an empirical paper published by Koch et al. in the same journal in May, in which they criticise the overestimation of abortion rates in Colombia by Singh et al. The Singh et al. response strongly disputes the criticisms raised pointing out that the method used to estimate abortion rates (that includes subjective opinion) has been widely used by their group in many countries. Not in and of itself a very convincing argumentum ad antiquitatem, appealing to historical practices. It is well known that all that is ancient is not necessarily wise and methods are not necessarily valid or reliable once used often. Having said that if the studies are published in decent peer-reviewed journals then that is at least something. Nevertheless, the authors do seem to concede that the method is in development and only provides estimates, not exact values. The question is whether estimates that even partially include subjective retrospective recall are robust enough for use in epidemiology.
Singh et al. go on to say that the method does not solely rely on subjective response but hospital records are used where available. They then criticise the Koch camp for misrepresenting their method on a number of grounds including that the sample was convenience and not randomised and the staff who participated were not knowledgeable enough to give accurate estimates. Secondly, the paper argues, the alternative method of estimation provided by Koch and colleagues is flawed. The response, out today, essentially reiterates the problems of a reliance on retrospective opinion survey and defends a method of utilising known statistics from countries like Spain and Chile as being standard epidemiology practice: "Strictly speaking, we have simply replicated a standard epidemiological model, using two standard populations, such as Chile and Spain, recognized for the high quality of their vital and health statistic records, to apply known rates and magnitudes to the official vital records of Argentina, Brazil, Chile, Colombia, Guatemala, Mexico, Peru and Dominican Republic." Whether or not these more objective methods of estimation do a better job than the opinion method remains to be seen. Sing et al. purport to have applied the method to data in other countries and report that the number of women hospitalised for abortion complications was overestimated by 99% for Brazil and 38% for Mexico.
Dr Sedgh defends the method robustly saying, "The proportion of all women having abortions who receive facility-based treatment for complications is obtained through a Health Professionals Survey (HPS), which is conducted among experts who are knowledgeable about abortion provision in the study country and who can estimate the proportion of women who develop complications and the proportion who receive treatment for them…. Substantial amounts of time and effort were invested to identify the best informed experts, who were selected on the basis of their professional affiliation, training, experience and expertise on the topic. [In Colombia,] most of the respondents (75%) were medical professionals; the rest were nonmedical health professionals who offered a broad community-based perspective….HPS respondents are required to have broader knowledge about abortion provision and complications, both inside and outside facilities."
Even if we are indeed seeing a problem of overreporting of abortion due to selection or recall bias in the opinion survey, the question then has to be why? Why would individuals tend to over rather than under estimate events like abortion? Dr Calhoun believes that, " The myths of maternal deaths due to illegal abortion have reached the level of urban legend. Only recently we have available reliable medical records and epidemiological data that made possible the research to provide the true facts regarding abortion in areas where elective abortion is illegal." Dr Koch notes that, "It is important to note that statistical counts based on prospective records of elective abortions is substantially different of studies based on self-reported abortions. In the first case, abortions are registered when the abortion procedure has been completed in a hospital or abortion facility. Thus, abortion figures will depend of the quality and integrity of the registry. The number of abortions may be under-reported especially when some facilities providing elective abortion services do not report regularly all abortions carried out in its dependences. In the second case, that is, studies based on self-reported abortions of women or estimates based in opinions of selected informants, abortion figures may be over- and under-reported depending of the methodology utilized."
The converse issue, of course, is an underreporting of abortion and related complications that will escape the records in hospitals and clinics. Dr Koch believes that this is unlikely to adversely affect estimates for the following reasons:
"Regarding the problem of underreporting elective abortion figures in the federal District of Mexico after abortion legalization, we think it is probable that not all abortions carried out in private clinics are being reported. However it is very unlikely that the level of underreport overpasses an extreme case of 100%, mainly because most of Mexican women (over 70%) use public health facilities that are free of cost. Part of the Mexican population prefers using private health facilities by their own monetary cost. Thus, probably the underreport rates would follow the same trend, mainly explained by the proportion of women seeking abortions in private clinics. For example, if we imagine a underreport rate of 100%, approximately 50% of abortion should be conducted in public health facilities and 50% in private clinics. Thus, it is very unlikely that in private clinics the number of abortion surpasses the number of abortions registered in public health facilities." Koch goes on to say that the way the ICD codes are organised, medics have the option of choosing a generic case of death such as ICD-10 O05 'death due to other abortion' and so can avoid administrative difficulties if they are living in an abortion prohibitive area."In fact, usage of these ICD codes allows safeguarding both professional and patient confidentiality when physicians suspect an illegal abortion. Moreover, nowadays in most countries, physicians are subject to legal sanctions if they are found guilty of distorting or misclassifying actual causes of death."
Dr sedgh, from the Guttmacher Institute believes otherwise however, "It is well-known that the official abortion records in Mexico DF only include legal abortions that take place in public facilities. Official records exclude the significant number of legal abortions that take place both in private facilities and outside of facilities entirely. Many women in Mexico DF obtain abortions outside the public sector for a variety of reasons. These include lack of information of availability of abortion services in public facilities, proximity of private facilities, and desire to avoid stigma associated with abortion. Some women turn to private providers and many obtain medical abortifacients from pharmacies or markets and self-induce at home with drugs like misoprostol, which is relatively safe and offers significantly more privacy. In short, government statistics on abortion represent are widely known to represent a fraction of all abortions taking place in Mexico DF, even according to Ministry officials. In contrast, the estimate based on the Guttmacher methodology account for all abortions taking place in Mexico DF."
So what of the disease burden of maternal mortality when compared with all diseases? How big of an issue is it in reality? Dr Calhoun stated that, "while in poor and low income developing countries maternal mortality continues to be one a leading cause of death in women of reproductive age, in middle and high income developing countries cardiovascular diseases, malignant tumours and external causes such as suicide and accidents are substantially more important than maternal deaths." Dr Koch added that, "for example, in the case of Chile, the relative importance of maternal deaths to the total causes of mortality in women of reproductive age is lower than 0.01%. However, it is important to point out that because most of maternal death causes are preventable and motherhood it is a fundamental value in practically all nations around the world, any maternal death is considered a tragedy."
In summary maternal poor health is a major concern in poor and low income countries and abortion accounts for a small yet important amount of maternal mortality overall. In terms of the UN MDG, number 5, what are the best ways to focus our efforts to reduce MMR? Dr Calhoun suggests that, "the focus for decreasing MMR is to provide better perinatal care in the area of hypertensive disorders in pregnancy, better care for postpartum haemorrhage, and provision of better obstetrical services.", while Dr Koch continues, "... the progress on maternal health in developing countries is the result of several factors whose importance will depend of the state of human development, the level of poverty and the current maternal death rate in each country. In general terms, these factors include an increase in the educational level of women, complementary nutrition for pregnant women and their children in the primary care network and schools, universal access to improved maternal health facilities (early prenatal care, delivery by skilled birth attendants, postnatal care, availability of emergency obstetric units and specialized obstetric care); changes in women’s reproductive behaviour enabling them to control their own fertility; and improvements in the sanitary system (i.e. clean water supply and sanitary sewer access)."
As is usual with most investigations, the process ends up throwing up more questions than answers. On the one hand abortion practices seem not to be a major contributing factor to maternal mortality, at least in relatively well-off countries and legislation, or liberal law doesn't appear to have a large effect on mortality. On the latter point Dr Sedgh believes that, "Restrictive laws do not have the impact they intend to have. Alternate means to reducing the incidence of unintended pregnancies and the consequences of unsafe abortion are greatly needed. To deny that these abortions occur is not the pathway to addressing the problems they represent." On the other hand, can we really not sympathise with women wanting abortion after rape or incest? But what about the negative impact of abortion itself? Koch and Calhoun argue that the experience of abortion for women can also negatively influence mental health, "A recent meta-analysis led by Priscilla Coleman (The British Journal of Psychiatry 2011; 199:180-186) shows 81% increased risk of mental health problems, and nearly 10% of the incidence of mental health problems was shown to be attributable to abortion." It is a complex issue and yet it is precisely because of this fact that measurement needs to be as accurate as possible. Are there overestimations of abortion rates? More independent research is needed, particularly comparing opinion survey in areas where full medical records are available but the weight of evidence thus far appears to indicate yes. So what you might ask? Well little progress on maternal health and MMR could cause commissioners of services, governments etc. to abandon programmes and initiatives that are in fact working, falsely believing that they are not. Having said that, more work is needed to ensure that progress continues to be made and that women do not suffer injury and death unnecessarily. If legal status is largely unrelated to maternal health there is a good argument to be made to relax laws in order to minimise the risk of unsafe abortion and increase transparency in epidemiological terms. The debate will continue of course. and it has to be noted that moral and ethical arguments have not entered the discussion here. The focus has been on methodology. But, the elephant in the room of course is the political, ideological, moral and religious influence on the science which is often ignored. But as I said in the introduction this piece is concerned with scientific method.
All comments welcome.
*this finding appears to be reported as a positive relationship in the results section of the paper in The Lancet