OK, here's a simple way to measure how violent things have gotten in Iraq. Calling around this week to five English-speaking Iraqis. Every one of them knew somebody who's died in the violence since the US invasion. A doctor in Baghdad, who's active in politics, told me he knows over 300 people who've died. A guy in Babylon says it's pretty safe where he lives, but his cousin was shot by Americans when he got too close to them on the road. A medical student's cousins died from an IED exploding in the street. And he has friends who died too. One of them was kidnapped. Then the cops were trying to save them, then accidentally he was shot. Then another friend that I had in college, he was in my grade-- it was last year-- he was in his car, driving to college, and he was suddenly shot by unknown men. Just today told me, when he went to the store near his house, he saw three people were killed. No one knows why. Ali, a bookstore owner in Baghdad that I talked to picked up a list of people he knows who've died. Friends, relatives neighbors. The situation, no matter how much I don't talk about, it cannot described how it is. How often will you hear that a friend or a friend of a friend or a relative has died? Is it once a week? Every day, believe me. No. Every day? I swear. Just two hours ago a neighbor has been killed in our neighborhood. He was shot dead. Shot dead. How, why, where was he? He was with his wife in his car, and some gunmen pulled over and just shot him and gunned him down because he's a Shiite. That's horrible. I saw three similar incidents-- getting a man killed in front of his wife and two children. The oldest girl is about five years of age. And she was shouting, oh, daddy, please daddy, don't die. In front of my eyes. I don't know these people, but this happened in front of my eyes. It's like you're getting used to it. You hear people die. OK, people die. It's at the point now where if you go up to a dead body and try to clear it away, you're in danger of getting shot yourself. It's at the point now where Iraqis are getting their names, addresses, and phone numbers tattooed on their bodies. So if they're killed, their bodies will be identified and returned to their families. People I talked to said they're scared all the time. Everybody said that it was better when Saddam was in power. Not that they like Saddam. Mohaned, an architect, described the choice between Saddam and the current situation in this way. They're two bad choices, I must say. Saddam's staying in power is not a solution. Yeah. And there was no light at the end of the tunnel, so to speak. But today there is no tunnel. Seems like every day we hear these stories and numbers from Iraq. Car bombings, more deaths, sectarian killings. But today on our radio show, we try to understand what you would think would be one of the simplest facts about the war, once and for all. From WBEZ Chicago, it's This American Life, distributed by Public Radio International. I'm Ira Glass. Our program in three acts today. Now, in the second half of the show we have a very unusual recording of US forces trying to make things right with Iraqis, after the US accidentally killed 12 civilians. In the other acts of the show we talk about a number, and in one of the acts a small news story that nobody ever really seemed to take very seriously at the time. And what was really behind that story. Stay with us. Act One. Truths, Damn Truths, and Statistics. Well, the number that we're going to talk about right now-- and we are going to talk about a number-- the number is how many civilians have died as a result of a war. Fact is, nobody is even pretending to have an accurate count. Iraqi Ministry of Health for a while, early on in the war, was compiling morgue figures from across the country and making them public each week. But that practice was stopped. There's also a privately run nonprofit web site called iraqbodycount.net, which keeps a running tally of confirmed deaths in Iraq. That's as opposed to an estimate of all the deaths in the country. And there have been two studies published in the British medical journal, The Lancet. One of the studies came out just a few weeks ago. The other came out two years ago. And both of these studies arrived at casualty numbers that are so high, they shocked many people. Right now, in this first act, we're going to return to a story that one of our producers, Alex Blumberg, did for our show a year ago about that first Lancet study. Later in the show, in act three, we're going to look at the study that just came out. Anyway, here's Alex. Everyone will tell you counting casualties in wartime is hard. First of all, you need to do something called a large-scale mortality survey. And second of all, you need to do it in the middle of a war zone. The first attempt ever made in Iraq was this Johns Hopkins University study published in The Lancet, a British Medical Journal, in late October 2004. A couple of days before the US presidential election. It concluded that probably a 100,000 Iraqis died as a result of the war. This figure was astonishingly high. 10 times higher than any other casualty estimates at the time. Because of this, and because the study itself got almost no traction in the press, I remember thinking at the time it came out that it was probably bogus and slanted. I'm guessing a lot of people, if they heard about the study at all, felt the way I did. But recently, in trying to find out how many Iraqis have died in the war, I've learned more about the Lancet study. And the more I learned about it, and the remarkable story of how it was done, the more likely it seems that the 100,000 figure is actually the best estimate and possibly low. Before the Iraq study, the main thing I was known for-- and that I'd testified in front of Congress for-- was documenting how many people had died in the war in the Congo. This is Les Roberts, the lead author on the Lancet study. And one of a handful of scientists in the world who could be called an expert in counting war dead. In the Congo study he found that 1.7 million civilians had died from the war, a figure cited by Colin Powell when he was Secretary of State and Tony Blair on the floor of the British Parliament. Les has also done studies in Burundi, Rwanda and Sierra Leone. To a guy in Les Robert's line of work, the war in Iraq had a number of unique and interesting things that deserve study. The main thing that distinguished this war was that the military took unprecedented care to avoid civilian casualties. Almost 2/3 of the bombs dropped were precision guided, as compared to just 8% in the first Gulf War and 0% in World War II. They limited daytime strikes and avoided civilian infrastructure like power and sewer plants. Compare that to World War II, where American forces firebombed entire cities as part of the military strategy, killing up to 100,000 people in Tokyo alone. And upwards of half a million civilians in Europe. And you can see why George Bush called the Iraq War one of the most quote, "humane military campaigns in history." The lesson here that often it's not bombs and bullets that kill people in war, it's the other things that happen when society falls apart. Clean water and medical supplies get scarce. In a lot of the studies he did in Africa, diarrhea killed more people than weapons did. Women can't get to the hospital to deliver their babies so infant mortality rates go up as well. It took Les a long time to get to Iraq and see if the same things were happening there. First, an Iraqi doctor who he'd planned to work with, died in an auto accident. Another social ill, by the way, that tends to increase during wartime. And then insurgent violence spiked. It wasn't until August of 2004, five months after he'd originally planned to go, that he finally landed in Amman, Jordan. He had $24,000 in foundation money in his pocket, his passport, and a letter of invitation from the Iraqi Ministry of Education. He found a driver, a retired Iraqi army officer named Wahid, who agreed to take him to Baghdad. Problems started at the very first checkpoint, on the border between Jordan and Iraq. Well, he takes my passport, he takes my letter of invitation, and he goes and he comes out just a few minutes later and he is just terrified. Turns out he bumped into a former friend of his from his military days, and he had pulled out my passport in front of him. And his friend just blanched and pushed the passport back into his pocket and said, "You have an American here. Are you crazy? Don't let anyone see that. Just get the hell out of here and don't let me see you again. And oh, you idiot." Fortunately for Les, Wahid he was something of a Han Solo figure. An unenthusiastic but talented smuggler, who didn't look for trouble but didn't run away from it once it found him. He talked his way through that first checkpoint. And we drive up a couple of miles. And he pulls off the road, behind this abandoned old gas station. And in the upholstery of his car, he's got hidden another pair of license plates with a different color. And he's got another registration form to go with those license plates. So quickly, he gets out and he changes his license plates. And he says, "Look, you must lie down. You must stay hidden." And so I spent the next eight hours lying on the floor. And we actually had to go through two extra points where they stopped and looked around in the car. And he chatted with folks and here I am, I'm lying down behind the back seat on the floor. So when they stopped and looked around, you were actually hiding from them? That's right. And were you scared? That's a funny thing. I had consciously made the decision that it was worth trading my life for a chance at getting a realistic estimate of how many Iraqi civilians have died and how they've died. So I was quite at peace with the notion of dying when I went. Les finally made it to Baghdad, where he met, for the first time in person, his Iraqi co-researcher. The man with whom he'd be working for the next month. His name was Riyadh Lafta, and he was a doctor of community medicine at Al-Mustansiriya University, in Baghdad. Riyadh had hired a team of researchers, mostly doctors from his university. All of them native Iraqis, but fluent in English. Let's pause here a moment to talk about their methodology. Because when the study came out later, a lot of people wanted to believe that it was flawed or biased. In fact, the survey team used a standard methodology for measuring health and mortality over a geographic area. It's called a cluster sample survey, and it works like this. Using the most recent census figures available on Iraq, the team made what was essentially a map of the population. They then used a random number generator to pick 33 points on that map. Baghdad was the biggest population center, so it got several points by itself. But the other points were spread all over the country, from the Kurdish north to the Shiite south. From small towns to big cities. Once they picked a town though, the team still had to figure out who the interview there. Here again, they worked hard to leave everything to chance. Using GPS units, they would drive around the outskirts of the town and store the coordinates, creating a rough outline of the town border. They would then generate a random point within that border, drive to it, and interview the 30 nearest households. It was such a commitment to random sampling that the first few times the team did it, even the researchers Les and Riyadh were working with found it obsessive. It was very annoying to them, because here they are in the car. they're out there feeling like they're at risk. And they'd be driving around for a long time to get to the extremes of the city and draw their map before they interviewed the first house. They're driving around and not getting any work done, they felt. And this is all just to make it as scientifically valid as possible, right? This is a way of picking houses without any sort of preference for safe neighborhoods, dangerous neighborhoods, near the highway, far from the highway. It was a way of sort of transcending human laziness so that, in essence, every household in Iraq had an equal chance we would visit them. And that is, in essence, the definition of random. The survey went smoothly, at least for the first couple of days. People, it turned out, were much more willing to answer questions, even to provide death certificates as verification, than the researchers had initially thought they'd be. In fact, the trouble, when it came, came from Les himself. Must be about the fifth day I was out with them, the eighth cluster I attended. I and two of the interviewers were up in a town to the north called Balad. And there was a huge picture of the cleric Sadr as he rolled in to Balad. So clearly it was an anti-coalition city in a big way. Sadr of Sadr militia. Of the Sadr militia. That's right. And as fate would have it, the first or second door they knocked on was the governor's house. And so somebody calls the police. Les watched from the car as the police took the two researchers, both doctors. One a dignified man in his fifties and the other a single mother, and drove them away. He was terrified that somehow the police would find out that they were working with him, an American. But he could do nothing but sit in the parked car and hope no one discovered him. I had done everything I could to be invisible. I wore boring Iraqi clothing. I had dyed my hair black. I had grown a beard so I would look right, but it still didn't look right. They had made up a fake business cards that said Doctor Abdul Salaam, that I was from Bosnia. Because that would explain me being a blue-eyed, non-Arabic speaking, but I could still be a Muslim and that would make me OK. And I was just so worried that sitting there for an exorbitant length of time would draw attention, that I put the back of the passenger seat down and I sort of laid on my side to pretend I was asleep. So I wouldn't have to speak to anyone if anyone came to the window of the car. And I had probably been lying on my side for about 20, 30 minutes. And these two little kids, they might have been 10, they came up to the window beside where I was. They stuck their head in the car, they looked around-- and I'm pretending to be asleep-- and they said to me in English, hello mister! And even with my eyes closed, pretending to be asleep, there's just no way I could pretend I was an Iraqi. And there was no way around it. So that was a pretty just horrifying experience all around. And I'm wondering, have I gotten these two lovely interviewers arrested or killed. And after an hour or a little more, a car brought back the two interviewers. And they went right back to work. They didn't come to the car. They didn't look at us. They didn't acknowledge us. They just went right back to work and finished out the 30 houses randomly picked in that neighborhood, and off we went. But after that day, no interviewer ever spoke to me again. Not in person. Riyadh and Les decided that for everyone's safety he should lay as low as possible. So for the next 16 days straight he didn't leave his hotel. To pass the time, he crunched the numbers that the survey teams were calling in to him every night. The surveyors were getting basically two pieces of information from each household. How many people in that household had died in the 14 months before the invasion. And of what and when. And how many had died in the 17 months after the invasion. And of what and when. By the time the teams had completed their 32nd out of 33 clusters, over 900 households and over 7,000 people, the results were pretty shocking. The death rate itself had gone up about 60%. A large increase, but one that Les had expected from his other surveys. The shocker was how people were dying. For the first time, in any of his surveys, the leading cause of death wasn't disease. It was bombs and bullets. In the 32 of the 33 clusters sampled, 21 people died of violence. That's compared to just one violent death in the period before the war. Of those 21, two people died in firefights where it was unclear where the bullet came from. Three were killed by insurgents, or Saddam loyalists. Seven died from criminal violence-- carjackings, revenge killings, that sort of thing. And the biggest number, nine, were killed by the American-led coalition. I should mention that only three of them involved guys with guns. All the rest were helicopter gunships, and bombs from planes. So it's not about individual soldiers doing bad things. In fact, two of those three cases, when soldiers shot civilians with their guns, they actually went to the houses of the decedents and apologized to the families. So there's no evidence here of soldiers running amok. There's evidence here of a style of engagement that probably has relied very heavily on air power that has resulted in a lot, a lot of civilian deaths. I was at a presentation last November, and a pentagon spokesperson said that they've dropped about 50,000 bombs in Iraq. 50,000 bombs. Very, very small fraction of them would need to miss their target or be based on bad information to explain 100,000 civilian deaths. At the end of three weeks, there was only one more cluster to survey. The team had saved it for the end, because it was the most dangerous one, Fallujah. Remember, this is September 2004. Insurgents control the city. And it's basically under siege from the Coalition. They're shelling it regularly. It just seems crazy to go there. And I said to Riyadh, "Riyadh, we have been to 32 of our 33 picked neighborhoods. We actually only thought in the end we would get to 30. We'd aim for 30 and pick 33, with the thought that 10% of places would be too unstable for us to get to. So we've done better than we expected. We have a terrible story to tell. The mortality is way up. Whatever you find in Fallujah is not going to change the story. Think of what we're going to gain. We're going to gain nothing. And he said, "God picked those random locations. God wants me to do this work. I must do this." And we went back and forth and back and forth and back and forth. And I was brought up Catholic and I had never really thought about it or understood it until that moment in time. But in my head, I actually sort of build up a weight. What's the likelihood of something bad going to happen to these guys, and how bad is that? What's the likelihood of something good coming from what they do, and how good is that? And I sort of put a weight on each of them. And I spoke with Riyadh. He actually did not have the capacity to do that, because for him, doing God's will and this work were inseparable. He couldn't separate out risk, because that was separating out sort of faith. The more we spoke, the more I understood that on some very, very fundamental level that we couldn't communicate with each other about our motives here. And in the end he went. Only one other interviewer agreed to go to Fallujah with Riyadh. A doctor who had relatives there he wanted to check up on. Their car was stopped three times on the way into the city. Heading to the random spot, they saw devastation everywhere. Houses were bombed. Rubble lay in the streets. The block they stopped on was no different. They had to visit 52 households to get the requisite number of interviews. 23 homes were either temporarily or permanently abandoned. Neighbors said that in the abandoned houses most people had died, but this data couldn't be substantiated. So it wasn't even included in the survey results. In the 30 households they did survey, there were 53 deaths. 52 of these were violent deaths. All but one caused by Coalition weapons. 24 of the people killed by Coalition bombs and bullets (SUBJECT were children under 12 years old. And with that the survey was over. Five days and counting. Tonight, the newest polls, the latest trends, and breaking developments from the campaign trail. On America's News live. This is Fox Evening News, October 28th, 2004, on the day of the results of Les's survey, that just shy of 100,000 Iraqis had died as a result of the war, were released. Les had not even considered the Fallujah data in coming up with this number. Fallujah had so many deaths, it was too much of a statistical outlier to even include. Fox never mentioned the study. Neither did ABC or CBS. The only national network that carried the story was NBC for 21 seconds. Tom, thanks. And we begin here with Iraq watch tonight, and one measure of the high cost of war. A new study from Johns Hopkins University estimates that 100,000 Iraqi civilians have died since the start of the war. The majority as a result of US air strikes. This is a much larger figure than some previous estimates. The Pentagon had no comment on the number, but said it had taken great care to prevent civilian deaths. And there is-- Morning Edition and All Things Considered on NPR devoted 45 seconds to the story. It didn't make the front page of the New York Times, the Washington Post or any national newspaper. The Iraq study had provided information about the war that up until that point no one had been able to provide. The number it gave was much higher than anyone would have expected. It was just as accurate as Les's previous studies in Africa that he had done using the exact same methods. And which were widely reported in the press and quoted by lawmakers. His Congo study was page one in the New York Times. The only differences with this study where that he'd risked his life to do it and it was about Iraq, which, if anything, should have made it more interesting to the media. So why didn't it get any press? Partly, it was the timing. The study came out five days before the US election, and so the media was pretty preoccupied. Plus, there was a suspicion that the team had timed the release of the survey specifically to influence the election. A suspicion that Les didn't really help dispel. He said to an AP reporter about the study, quote, "I emailed it on September 30th, under the condition that it come out before the election. My motive in doing that was not to skew the election. My motive was that if this came out during the campaign, both candidates would be forced to pledge to protect civilian lives in Iraq. I was opposed to the war, and I still think the war was a bad idea. But I think that our science has transcended our perspectives. As an American, I'm really, really sorry to be reporting this." One desk editor at a national news organization told me that when the study came out, he sent an email to one of his colleagues saying, The Lancet had, in the past, published some studies with a political slant. But that this study seemed sound, and maybe they should report on it. Then he saw Les's comments, and he didn't follow up. This is exactly the type of story that those who believe the media has a liberal bias love to pounce on. And so in essence, if the research turns out to be flawed, this desk editor's organization gets the heat for it. He had a very small window of time during a very busy news cycle, to decide whether the study was legit or just an angry and easily debunked researcher pushing an agenda. And Les's comments seemed to be all the evidence he needed. And there was one other thing that made it easy for the media to dismiss the report. A researcher at Human Rights Watch, who himself had done studies of civilian casualties during wartime, said he didn't believe the study. The researcher's name was Marc Garlasco, and he told a reporter for The Washington Post, quote, "The number seems high to me." And quote, "It seems like a stretch." I was actually on the Long Island railroad when he called me. It was some time in the evening, and I had yet to read Les's report. This is Marc Garlasco. He said he told the reporter from The Post that he hadn't read the study. But the reporter said he really needed a quote, and could he just respond to the number. Garlasco's quote was cited elsewhere and he appeared on CNN, although none of the study's authors were interviewed on CNN or any of the major networks. Here's what Marc Garlasco says now. First of all, I'm not a statistician. I know absolutely nothing about it. And when I then went and spoke to statisticians they said, well, you know the method that he's using is a really accurate one. This is something that we use in studies all throughout the world, and it's a generally accepted model. And that kind of made me think about it. Think about my prejudices going into reading his report. Because I had been on the ground, in Iraq, immediately after the war. But I also had taken part in the targeting for the war. OK, let's just stop here for one minute. You heard what he said. He'd taken part in the targeting for the war. Get ready, because this story's about to take a turn. Marc Garlasco isn't your typical human rights advocate. Well, I worked in the Pentagon almost seven years. And my last job there was chief of high-value targeting on the joint staff. And basically that means that I was one of many people that was involved in the tracking and attempted killing of Saddam Hussein and all those people in the deck of cards. And I would sit there with my compatriots, and we would put x's on buildings one day, and the next day those buildings are gone. So you were literally in this last invasion? Absolutely. I was involved in the war planning. In January of '03, I was involved in the final targeting of Iraq. When we put the final target list together, and of course those got brushed up as we got closer to the war. During the war, I was working 18 hours a day, at least, in the Pentagon. Putting in hours, trying to get and kill Saddam Hussein and others. And after Baghdad fell and then on April 11, I walked out of the Pentagon. And it was a Friday. And then on Monday morning I walked into Human Rights Watch, and suddenly I'm now a human rights advocate. And got on a plane and flew to Iraq to see my handiwork. That literally, how soon after? Literally, it was just the next week. Got onto a plane and went to Iraq. And I was standing there in craters that I had helped cause. Marc doesn't see moving from the Pentagon to a human rights nonprofit as the 180 degree flip most people might. He says all he's ever wanted to do is fight bad guys and both organizations do that, just in different ways. He'd been thinking about leaving the military before the war began, and he hadn't supported the war himself. But he stayed through the fall of Baghdad, because he knew the target set better than anyone else. And he figured if there was going to be a war anyway, it might as well be him targeting the bombs rather than someone else who might not know or care as much as he did. The thing that finally prompted him to leave, he says, didn't have anything to do with the war. His wife got a great job offer at the Bronx zoo, and they'd always wanted to move back to New York. When Marc went with Human Rights Watch to Iraq, it wasn't to get a comprehensive count of civilian casualties. His mission was to look at specific attacks and see which kinds of attacks caused high civilian death tolls. Because Marc had planned many of the strikes he was now going to investigate, it was a little complicated. There was the attack on Chemical Ali in Basra. And I'll never forget sitting in this tiny cubicle, in the bowels of the Pentagon, watching it on the television as we had the predator overhead. And you're watching this black and white screen, because it's a night shot. And anything that's white is hot and black is cold. And we're watching people walking in front of it. And all of a sudden, this building just erupts and was gone. And we watched as bodies flew out of it. And you could see the legs kicking in the air like rag dolls. And we just erupted in cheers. And we were ecstatic. Here we are. We killed Chemical Ali. This is great. And what is it, three weeks later I'm standing in the crater with this 70-year-old man who's got tears in his eyes. And he's telling me how 17 members of his family, including his grandchildren, were killed. and I still feel very mixed emotions about the whole situation, the whole experience. What are those mixed emotions on the one side and on the other? Well, on the one side, I feel like I took part in this wholesale slaughter of this guy's family, which is very difficult to swallow. But on the other side, I know that we truly, truly did what we could. We were going after some very bad people-- war criminals. Chemical Ali had gassed the Kurds. He was singularly responsible for thousands of deaths. And so, he was certainly a legitimate military target. But I think this just goes to show how difficult the job really is. This is one of those strikes where we did everything right. Where we thought we had the bad guy. Where it was weaponeered correctly. And yet, it just was the wrong place to hit at that time and people died for it. The attack had hit the intended buildings, but it also destroyed the two neighboring buildings. That's where the man's family had died. Also, Chemical Ali hadn't been in the targeted building anyway. It's unclear who died there. Marc went to lots of places in Iraq he'd studied on maps and aerial photographs and heard about from defectors. And there's no way around this. After all those years of imagining these places, what they must be like, it was exciting to actually be there. I was walking through bunkers that I knew about. I went to Saddam Hussein's bunker. I went to his family's bunkers. One of my favorite moments was when I actually met one of the bunker builders and hired him as a translator. And he took us into Sajida's palace. Sajida was Saddam's wife. And we knew that there was a bunker under the building, and we had targeted it and dropped a weapon into it. And he took me in. And we go into the building. And I'm seeing the inside for the first time, which had before only been described to me by defectors. And here it is and it's this picture that had been painted in my mind. And we get there and the guy says, now I will walk you down to the bunker. And we walk down to it, and we get to the bunker. And when we look down on it from the top, there's a whole as the Penetrator went in through the four floors, straight down into the bunker. And he looks at me and he says, whoever did this was a very smart man. And I lost it. I just completely lost it. Because you're like, I did that. I was like, hey thanks. I appreciate it. The military denied my request to talk on the record about civilian casualties. But Marc Garlasco says that civilian casualties are one of the primary factors he and his colleagues considered when planning the war. Say he had a target he wanted to take out. The headquarters of the Iraqi Secret Service maybe or one of Saddam's palaces. He'd work with the weaponeering guys to figure out how many and which bombs to use, and then-- Once that's established, they'll work up these collateral damage estimates and tell you, OK, in this strike 10 people are anticipated to be killed. Civilians or 20 civilians or whatever. And in this war in Iraq there was a magic number. And the magic number was 30. And for any target where it was anticipated that 30 civilians or more would be killed, it required the signature of either the President or the Secretary of Defense for that strike to actually occur. How was that magic number arrived at? Do you know? I have absolutely no idea how the magic number came to be 30. A lot of times, when the collateral damage assessment came back too high, they'd try to get it lowered. For example, a strike Marc planned early on in the invasion. An Iraqi division was holed up in a big multi-building convention center in Baghdad, which unfortunately was right across the street from a hospital. Now because of the amount of guys there and the construction of the buildings, we knew that they needed to use 2,000 pound bombs. The problem with this is a 2,000 pound bomb has a very large, destructive radius. And it certainly would have enveloped the hospital. But there are things that you can do, even when you're dropping large munitions, to reduce civilian casualties. One of those is to change the angle of attack. And so imagine if you will, a plane is coming in and drops bombs at such an angle that they actually push the debris away from the direction of the hospital. Additionally, you put a time delayed fuse on it. And, in this case, I think it was maybe five nanoseconds. Which is an incredibly short period of time, but it's enough that it allows the bomb to bury itself in the ground. And what this does is it basically lets the building implode. And it falls in upon itself and contains a lot of that blast and fragmentation damage that would come out and injure civilians or destroy some of the hospital. And then additionally you're using a penetrating warhead, so it's burying into the ground. So you're not just willy-nilly dropping bombs like in the Second World War. When I got there, I went into the hospital and spoke to the director and all the people in there. And nothing worse than a few broken windows. And I was like, wow, this is great. We did a really good job on this one. What got Marc thinking about civilian casualties in the first place, was a battle damage assessment he did after the war in Kosovo. He targeted the bombs for that war. And then afterwards, the military sent him over to see how well he'd done. He measured how often the bombs hit their targets. Whether they destroyed what they were supposed to destroy. Pretty much the only thing he didn't check the accuracy of were the collateral damage assessments. That's what got me. That's what really surprised me. At no point in time did we ever have to report back on civilian casualties. And so my question has always been, if the weapons worked correctly, if the targets were correct, shouldn't you also be asking were your civilian casualty estimates correct? Shouldn't that be factored into it, to make sure that your models are accurate? Because if your models are not accurate, what are they worth? Why do you even bother doing it? it's just throwing darts at a board at that point. Wow. Did you ever find out an answer to that question? No, but it's something that I keep asking the military now that I'm in Human Rights Watch. When I was there I was wondering why isn't it done. And now I asked them why isn't it done, and why don't you do it. And I guess the answer that I get back just hasn't satisfied me. It's look, we're still fighting a war in Iraq. It's really hard to do. Or it's very difficult to account for civilian casualties, for a variety of reasons. And you get the bureaucratic double-talk. And it's just not good enough, because I've been there, and I know that people care and want to do the very best they can. And they don't want to kill civilians. And here's an opportunity to really make a difference, and to show that you're doing your utmost best to make sure that you're upholding the Geneva Convention and not killing people unnecessarily. In talking to people in the military, off the record, I heard a couple of arguments against counting civilian deaths. First they say, it's not the military's job. What you're trying to do is win a battle. It could be a dangerous, and in the long run, counterproductive distraction to worry about counting all the civilians you accidentally kill along the way. Second, and perhaps more persuasively, they say no one would believe them anyway. Just ask Les Roberts. Even though Les's study didn't get much mainstream attention, it did provoke, like so many things these days, a bitter debate on the internet. The attacks came mainly, though not exclusively, from right-wing blogs. Several charges leveled at the study were simply untrue, and seemed designed to willfully muddy the waters. For example, there was a claim repeatedly made, both in the press and online, that the data weren't random because the researchers had been blocked from going certain places. Or had decided against certain places because it was too dangerous. This is simply false. A couple of people suggested that the researchers had gone to Fallujah, on purpose, to boost their numbers, even though exactly the opposite was true. Les had wanted to skip Fallujah altogether, and they hadn't even included the data in their final casualty estimate. Several objections had merit though. First of all, the study makes no distinction between combatants and civilians. Les actually acknowledged this in the study itself, and went to great lengths not to claim, as others did on his behalf, that the study was a measure of civilian mortality. Certainly some of the people the Coalition killed, they intended to kill. But half of all the casualties were women and children, so even in the unlikely event that 50% of the men who died were actually fighting us, it's still a large number of innocents. The critique that got most traction on the internet though has to do with something called the confidence interval. Let's take an election poll as an example. Candidate X is projected to receive 55% of the vote. What that really means is that he's projected to receive some percentage within two numbers, let's say 52% and 58%. That range is called the confidence interval. The confidence interval in Les's survey was very wide, between 8,000 and 194,000. It was this wide for a lot of reasons, but mainly because the sample is relatively small relative to the population. And because violent death, unlike death due to malaria or diarrhea, isn't very uniformly distributed. So you have Kurdish areas where mortality actually went down during the war, versus Fallujah, which averaged almost two violent deaths per household. Such a wide confidence interval means that, statistically speaking, Les's estimate of 100,000 dead isn't very precise. The number could be thousands or tens of thousands smaller or equally likely, bigger. But a lot of people made wrong conclusions from the wide confidence interval. They interpreted it to mean it was just as likely that 8,000 people had died as it was that 100,000 had. The online magazine Slate wrote, This isn't an estimate. It's a dart board. In fact, the likelihood follows a bell curve, with 98,000 being at the top of the bell, the most likely number. So actually, there's only a 2.5% chance that the number is 8,000 or below. But a 90% chance that it's 44,000 or above. Here's Les. A couple of people told me that that Sunday before the election their minister from the pulpit had said, this study in The Lancet was flawed and wrong. And my next door neighbor, who was listening to talk radio, spoke to me the day of the election. And she said, well I just heard on talk radio today that The Lancet study finding 8,000 Iraqi deaths was flawed and wrong. And so it was discussed, but I don't think it was discussed in a very sort of scientifically rigorous process. Clearly, the people on talk radio weren't attacking the study out of a commitment to experimental rigor. They were attacking it for the same reason that the news media was hesitant to report it. Because the very act of counting civilian casualties is political. The moral logic of war is this. We're willing to undergo x number of costs in lives, money, resources to accomplish some goal. The goal, we hope, will be worth it in the end. So assuming the goal in Iraq is good, is it wrong to kill 100,000 civilians? Saddam himself probably killed 230,000 of his own people. A number, by the way, no one seems to go out of their way to dispute. If you add the million or so lives he lost in the futile war he launched against Iran, 100,000 seems like a bargain in comparison. Maybe he would have gone on another killing spree, and this 100,000 is insurance against the later, far worse death toll. Or maybe 100,000 lives is worth it, if in the end democracy does blossom through the Middle East. After all, we killed far more Japanese with just two atomic bombs than, according to Les, we did in a year and a half in Iraq. If we don't count civilian casualties, we don't have to get into this kind of horrible math. And most of us don't want to. So instead, we leave it to the professionals. The military are the only ones who even try to come up with a formula, the collateral damage assessment. 30 dead civilians for one bad guy. For Les, he doesn't really care who counts, just so long as someone does. Under the Geneva Conventions, an occupying army's relationship to the occupied is roughly the same as a police department's relationship to its population. And in my hometown, if a policeman pulls out his gun and shoots six shots at someone, another policeman will come and try to find where each of those six bullets landed and decide, was this excessive use of force. And well, how can we say that we are really looking after the well being of the Iraqi folks, people, if we don't sort of go through some sort of minimal effort to decide what are we doing to them, and what can we do to limit the adverse consequences. One of the most surprising things Les discovered in Iraq is that, despite what everyone says about the difficulty of counting civilian casualties during wartime, it's actually not that hard. The survey teams got participation rates that most American pollsters would kill for. Only 5 of the 988 households the team surveyed refused to answer the questions. And people were able to provide death certificates over 80% of the time. That confidence interval, Les is sure that based on the results of the first survey and with a little more money-- remember, this whole thing cost only $40,000-- he could design a follow-up survey that would narrow that interval way down. We can count civilian casualties in wartime. We just have to want to. Alex Blumberg. Coming up, so a military that's trying as hard as it can not to kill civilians makes a mistake. What do they do then. Well, we have a recording of what they do then. Also, we take a look at the new Lancet study. The sequel to the one you just heard about. It just came out two weeks ago. That's in a minute. From Chicago Public Radio and Public Radio International, when our program continues. Well it's This American Life, I'm Ira Glass. Each week on our program we choose some theme, bring you a variety of different kinds of stories on that theme. Today's show, What's in a Number. In Act One of our show, we heard about the number of Iraqi deaths that occurred in the first year and a half after the US invasion. And this is no longer true in Iraq, but at the time, according to this Johns Hopkins Study we were talking about, the majority of deaths were caused by the US-led forces. Despite everything they did to avoid Iraqi deaths. Well, now we're going to move to Act Two, where we hear US forces trying to cope in the aftermath of some of those deaths. This is Act Two, Not Just a Number. The civilian deaths, in this particular story came at a unusually bad time for Captain Ryan Gist. He was the American officer in charge of the US Army presence in a section of the Nineveh and Salahuddin provinces in Iraq, a Sunni area. And this meant that he dealt with everything from seeing that the region had water and fuel to dislodging insurgents and winning over the local population. He faced all sorts of problems. On November 10th, 2004, for example, insurgents blew up most of the police stations in this area. And the next day, Iraqi Army and police simply stopped showing up for their jobs. One town called Atha was especially troublesome. The US had not made many allies there. And it was turning into probably the greatest threat to stability in our region. So to begin with, we knew we were going into a very hot spot. So we went in. And for weeks we'd go in there and we would try to talk to leaders, try and talk to people, and they were scared. The terrorists literally had a grip on everybody. And that is the challenge in Iraq. And what it takes is just feet on the ground, every day, going to different houses and just talking to people. And finally we found someone whose family had been killed-- every one of them had been killed by the terrorists within the last month-- and he wanted revenge. And he wanted to see his town free of this threat. So the guy gives him names and locations of people that he identifies as insurgents. And the army launches an operation to get those people. And this is where things went wrong. As part of this operation, the US forces were supposed to drop a bomb into a nearby field, just as a show of force. But instead, the bomb was dropped onto the house that the army was just about to raid, killing 12 people inside, including children. And nearly killing the US troops who were about to go in. That's according to an American photojournalist, Sheryl Mendez, who was there. In addition to the human tragedy of these deaths, for Captain Gist this could not have been worse. It was the hardest time of my life. And the most difficult part for us was in the weeks afterwards. We have to show that it was an accident. And that we are someone who can be trusted, and we are here to help them. The photographer, Sheryl Mendez, tape-recorded him as he went to the sheiks, who were the local leaders and to the police, as he went around trying to make things right. Here's one of those recordings. He's in a police station. I'm here for a couple reasons today. The first reason is to express my regrets to you. I feel much pain in my heart for what happened. We did not intend to hurt any innocent people here. We did not mean for any women and children to die. I had men hurt in the explosion as well. You're sitting there in a room with five, six, seven people that are definitely angry at you. They believe you did it intentionally. That hate you for being in the town in the first place. And I know that we cannot bring them back. Only Allah can do that. But I hope to be able to help with the healing process. I'm also here to ensure this does not destroy the relationship we've established with people of Atha. I know there's a lot to do to rebuild this relationship, and this is my first step. But it's also very important that I get all the facts here. Right now I have 12 names of those who were killed and three that were wounded. So I think I'm missing one here. It just seems like such an incredibly awkward thing you're having to do. I don't think awkward is at all the word you would use to describe something like this. It was possibly most emotional event of my life. Going out and dealing with these people in such incredible grief. Toman. Toman. Akmed. Akmed. Adur. Adur. Azisa. Azis. Aziseh. Aziseh. Aziseh. What happened after that? The way it works in the in the Arabic culture is, there's about a seven to ten day mourning period or grieving period after the death. So of course, I wanted to go in there. We're Americans. We want to go in. We want to fix it immediately. I wanted to go in there and make the payments to the families, and talk to them and express our regrets. So the hardest thing for me was I couldn't go into the town for seven days. So it was incredibly difficult for me. Because during those seven to ten days, you're thinking these people are deciding they hate us. And they're spreading word about how we killed these people. It's just the worst thing, from a propaganda point of view and winning over people's hearts. That's what your fear is. Well, that's part of the fear, but also I'm a human being. I'm an American. I want to get in there. And I want these people to understand that we are good people, and we're here to help them. Yeah. So really, at that point, I'm sure you can hear it in my voice there. Is that I know I can't fix it. I know I can't bring them back, but I want to make it better. But out of respect, we waited seven to ten days. We went to neighboring towns and villages and told them what was going on and got the message out. And I know the message got to the people in the town. But then the following three weeks we went in and did exactly what we were doing before-- it was eet with people, and begin to develop or rebuild relationships. And we continue to conduct operations, continued to pull bad guys out of the town. This is pretty much the standard way that he'd operated in these villages. He says, sure, there were some villages and local leaders who were sympathetic with the insurgents. But in other places, they were being bossed around by them. A couple of terrorists who intimidated the leaders of that village could effectively control the entire village. People were scared to leave their houses. Kids stayed in the houses. Kids didn't go to school. So when you went in, and you took those guys out. And you came back and said, hey, I just caught this guy, this guy, and this guy. And I found all kinds of explosives, threat CD's, beheading CD's in their house, they're gone. They're never coming back. And the most incredible thing was, you saw the change within a day. Within 24, 48 hours you would go back into that town, in a town where kids would not even wave to you or kids throw rocks at you. You'd go in the next day and they would be crowded around the humvees, just wanting to touch you or talk to you. In Atha, he says, people waited to see who was going to win, the US-led forces or the insurgents. Until there came a turning point. There was a distinct turning point. The turning point was when the sheik of the town came to see me. Which would never have happened before. And do you remember what he said at the beginning so as to explain? What do you remember of what he said? What he said was, I need your help. And that was it right there. I knew what was going to come after that. Just thinking about that accidental bombing, the way that it ended up working out, did that accidental bombing actually give you access to people and a way to meet with them, and a reason to meet with them that you might not have had otherwise? It wasn't that it gave us a reason to. We had a reason to, as we were going to be in the town, regardless. But it forced them to listen to us, because they were so hurt by the incident. And they began to understand what we were there for. And they eventually offered us forgiveness and asked for our help. And to this day, I probably have the strongest bond with that town than any other town over there. Captain Ryan Gist. He left the military in the Fall of 2006. He's now a law student at the University of Wisconsin. Though he still stays in touch, by email, with the Iraqis he knew in Atha. Act Three, The War This Time. So a couple of weeks ago, the Lancet published another study of deaths in Iraq by the same research team who did the earlier study. This time, the number of deaths that they came up with was seen as even more astounding than the first study's number. Over 650,000 extra deaths because of the war. 601,000 of those due to violence. This second study got slightly more coverage than the first study. A reporter even asked President Bush about it. Our producer, Alex Blumberg, examined the debate over this new study. And he called various statisticians and epidemiologists who looked at study. Well, if anything, the methodology is sounder in this study than it was in the last study. A lot of the critiques that were leveled at the last study could not be leveled at this study. For instance? For instance, one of the biggest critiques of the last study was that it had a very wide confidence interval. And in this study, the confidence interval is much narrower, because they went to a lot more places. In this survey they went to 1,850 households total. And the first study, they went to only 988 households. So the critiques that you've seen of this study, are there criticisms of this study or questions about this study that seem particularly compelling though? Everybody I talk to said that if you're doing this in a lab, you would do it differently. But almost everybody agreed that given that they were doing this in a war zone, the methodology was as sound as it could possibly be. And it's hard to imagine that any sort of bias that crept into the methodology could have accounted for a several hundred thousand person overcount. The fact of the number is so shocking to people who know about this stuff, that that's been the basis of criticism. And from groups that you would think would be sort of aligned with the Lancet's results. In other words, this group called Iraq Body Count. This is a group that's basically trying to do the same thing that these researchers are doing. Since the beginning of the war, they've been documenting every news report of where civilians are reported killed. And they've been keeping a running tally. And then, as the war progressed, and as they got more sophisticated, they started incorporating Baghdad morgue data into their into their tallies. And they've also started keeping hospital records. And the number they have on their website is around 50,000 civilians dead. That's their computation based on these methods. Based on these methods. Right, exactly. Which is less than a tenth of what this Lancet survey is saying they came up with. And they admit that they're doing an undercount. They admit that not every death is reported in the press, and that there's problems with the morgue reporting. So they know that they're under, but they don't think they're under by this much. So they came out with a detailed response on their website, and it's very compelling to look at. For instance. So one of the things that they do is they dig into the data. And so, they sort of go into the Lancet study and they say, on average, in the first half of 2006, according to the Lancet, 1,000 Iraqis have been violently killed every single day. With less than a tenth of them being noticed by any public surveillance mechanisms. They're not showing up in the morgue. They're not showing up in the press. Nothing. They're not showing up in the morgue. Or they're not being reported by the morgue. They're not being reported by the Ministry of Health. They're not being documented in press accounts. Right. And then they sort of go on further, because the Lancet breaks down how people die. Of gunshot wounds, which they say that's very likely. It's very possible that you could miss a lot of the gunshot wounds, if somebody is being killed execution style and dumped in a river somewhere, maybe they never show up. But there's a certain percentage who are dying of car bombs. And what they say is, car bombs people notice. And that gets reported in the press. But even according to Lancet figures, if you extrapolate, even the car bomb figures aren't showing up. In other words, if the Lancet study is right. We should be seeing a lot more car bombs in the news and we're just not. And we're just not. Right. And so that's one thing. And then the second thing they say is that there should be a lot more wounded people. Because in modern wars, for every person that's killed, there is at least three that are wounded seriously. That's kind of a general rule of thumb. That's a general rule of thumb based on lots of lots of studies of wars. So for every death-- so if there's 600,000 deaths, there should be some much greater number of wounded people who are showing up. And they do some sort of rough calculations. And they conclude that conservatively, if there are 600,000 people being killed by violence, there should be at least 800,000 people seriously wounded. But if you look at the hospital records all across Iraq, you don't see 800,000 people as having been treated for shrapnel wounds and legs getting blown off and all the things that you would expect to find for this level of violence. And did anybody who you talked to have any explanation for how these numbers could be so high? Well, there's two possibilities. One is that Iraq Body Count has more faith in the reporting mechanisms currently existing in Iraq than they should. And that, in fact, even though it's hard to believe that 800,000 people are showing up at hospitals and that's not getting documented, it might be. The other possibility is that the data is wrong. Either people lied to them and told them that people in their household had died, and then somehow fabricated death certificates to support that. That seems impossible. Because they were randomly selected. So it's impossible that that happened. Or the surveyors themselves concocted data. Said that people died in these households when, in fact, they hadn't. I don't think that happened. I think that would be such a huge conspiracy and such a cover up. And such a risky thing for all these people to do. Because Les was involved-- Les, the guy in the story-- was involved in both of these studies. And he did go there, and he did risk his life. Right. If you're going to make up stuff, why go door to door in dangerous areas? Why not just sit in your office and make up stuff? And if you're going to go door to door, why not just get the data? Right. And if you're going to make up stuff, why make up a number that even the people who are on your alleged side are going to argue with you about? That doesn't make any sense at all. If he was going to fake the data, he would have said, there was 300,000 dead. And then everybody would have been joining in and saying, this is a horrible thing. Right. And instead, he comes up with this much larger number. It just doesn't make any sense. So if you're going to lie, do a better job. If you're going to lie, make it a more plausible lie. Alex Blumberg. Special thanks today to Jerome McDonnell and WBEZ's show Worldview, where you first heard Les Roberts. Thanks Ben Shapiro and Beth Osborne Daponte, Fred Caplan and Michael White, Nancy Yousef. Laura Poitras, who has a new documentary about Iraq coming out called, My Country, My Country. Jamie Tarabay, Adam Davidson, and people at War News Radio, including Wren Elhai. This American Life is distributed by Public Radio International. This American Life is made possible by Volkswagen of America and the Volkswagen Jetta, encouraging listeners to stop stereotyping. Learn more at thejettareport.com. Support for our podcast, our free podcast, comes from audible.com, where you can download audio books, magazines, newspapers, and radio shows, including archives from the last ten years of This American Life. Available at audible.com/thisamericanlife or at the iTunes store. WBEZ management oversight for our program by Mr. Torey Malatia. Whenever he finishes a budget or correctly orders a lunch, he calls me into his office, leans back in his chair-- And he looks at me and he says, "Whoever did this was a very smart man." I'm Ira Glass. Back next week with more stories of This American Life. PRI, Public Radio International.