At a dirt test track near the Georgia Institute of Technology campus, researchers monitor a scale-model autonomous car as it drifts around corners at a blistering eight meters per second — equivalent to 90 miles per hour in a full-size vehicle. Pushing this car to its limits could help make full-size driverless vehicles more stable in risky road conditions.
This unique one-fifth-scale device is just one of many research efforts aimed at helping the autonomous vehicle revolution happen successfully and safely.
Self-driving cars are unquestionably coming, guided variously by radar, lidar, motion sensors, cameras, GPS, and plenty of onboard computation. Already, semi-autonomous prototypes are operating under controlled conditions in California, and speculation about future autonomy includes visions of commuters napping through drive-time, high-speed convoys of networked big-rigs, and a huge drop in accidents as robotic vehicles take over from impaired and distracted humans.
Yet these are only visions, where generalizations rule and few facts are established. At Georgia Tech, research focuses on the elusive but critical details of this phenomenon, as investigators from disciplines as diverse as industrial systems, design, engineering, computing, and psychology are developing a roadmap to robotic vehicles.
Researchers at Georgia Tech generally agree that a long period of adjustment, including generations of semi-autonomous vehicles, will be needed to reach completely autonomous transport on a large scale. Estimates of the time required vary from a couple of decades to more than half a century.
“Fully autonomous transport will require absolutely reliable navigation systems, major changes in highway infrastructure, and traffic control that’s synched to the vehicle, plus new fueling, insurance, financing, and manufacturing paradigms,” said Vivek Ghosal, a professor in Georgia Tech’s School of Economics, who studies the automotive industry. “Yes, we have prototypes, but the operationalizing of autonomy is still far away.”
A four-level model of the vehicular-automation process is now widely accepted. Level one denotes today’s driver-dependent cars; level two involves intelligent cruise and lane control with some automatic braking; level three indicates semi-autonomous vehicles that drive themselves but cede control to a human when conditions demand; and level four means fully autonomous with no driver controls.
Researchers at Georgia Tech, focusing on the gritty details, have spotlighted a list of complications that include:
- Human-machine interaction issues.
- Costly highway infrastructure changes.
- Unpredictable traffic effects.
- Conflicts between self-driving and human-driven vehicles.
- Guidance system reliability concerns.
- Vehicle ownership, liability, and business model shifts.
- Potential for major changes to the urban landscape.
This article takes a look at some of the research currently underway at Georgia Tech related to self-driving vehicles.
At the Georgia Tech Autonomous Racing Facility, researchers are studying a one-fifth-scale autonomous vehicle as it traverses a dirt track. The work will help the engineers understand how to help driverless vehicles face the risky and unusual road conditions of the real world.
Photo: Rob Felt
Many researchers believe semi-autonomous vehicles will dominate during the years or decades needed to sort out the complexities of fully autonomous transport. One key question: How can a semi-autonomous car or truck share driving responsibilities with people smoothly and safely?
At Georgia Tech’s Sonification Laboratory, Professor Bruce Walker pursues studies on the use of sound to convey information to people whose eyes are otherwise engaged — a surgeon focused on a procedure or a firefighter hampered by smoke. Walker, who serves jointly in the School of Psychology and School of Interactive Computing with a focus on engineering psychology, is testing interface approaches that would allow a human riding in a semi-autonomous car to search the Web or read a book while also staying reliably informed about vehicle operation.
One key research area focuses on situational awareness, explained Walker, a member of Georgia Tech’s GVU Center as well as its Institute for People and Technology. Drivers of conventional cars must be constantly aware just to stay in a lane, but in a semi-autonomous vehicle with features like advanced lane control, occupants could work and play without much regard for the road or other drivers.
“Inevitably, however, there are going to be situations where the autonomous processes — the computer vision, the guidance smarts, the GPS signal — will fail and control has to be handed off to a human,” he said. “The system needs to be able to keep the driver informed of any potential problems — my contention is that we can never let the driver lose situational awareness.”
Critical to maintaining this shared awareness is developing ways for a semi-autonomous guidance system to monitor itself — essentially, to maintain confidence in its own ability to handle what’s ahead. Under this approach, when a guidance system’s self-confidence falls below a certain point, the vehicle will alert the driver to take over.
To achieve effective vehicle-human communication, Walker’s team is testing a wide variety of background sounds. The researchers are weighing different approaches for conveying escalating levels of concern, from “all systems go” to “intervention needed.” The goal is to keep drivers in the loop without alarming them.
To assess how test subjects interact with multiple scenarios, the team is using three driving simulators, including a miniSim testbed devised by the National Advanced Driving Simulator group at the University of Iowa. This realistic setup combines advanced software with full-size control pedals and steering wheel, multiple plasma screens, and even a genuine car seat. Eye-tracking equipment and physical response monitors track a user’s interactions with the visual driving routines.
Panasonic Automotive, currently a dominant producer of automotive control systems such as cruise control, has been paying close attention to Walker’s work. John Avery, engineering group manager for the Panasonic Innovation Center in Georgia Tech’s Technology Square, acknowledges such research could be relevant to the company’s future activities in advanced driver assistance systems.
“It’s important that semi-autonomous cars drive in a way that makes humans comfortable,” Avery said. “Their driving style shouldn’t be so aggressive that they alarm the occupants, or so passive that people become frustrated.”
Developing Control Language
Driving simulators are valuable tools for autonomous vehicle research. In the Human-Machine Interface Laboratory, Wayne Li is investigating the role of interior components in the critical task of autonomous-vehicle control. He wants to know how control language — what people see, feel, and touch inside a vehicle — can make users confident even when they don’t have conventional full control.
In the Human-Machine Interface Laboratory, Researcher Wayne Li is building a driving simulator to investigate the role of interior components in the critical task of autonomous-vehicle control.
Photo: Rob Felt
Li is working with a Georgia Tech-built testbed consisting of parts of a 2010 Chevy Malibu mounted on a highly adjustable aluminum frame. The setup uses special software along with multiple screens and eye-tracking sensors to gauge user reactions.
The system, installed at the College of Architecture, is a joint effort of Georgia Tech’s School of Industrial Design, where Li is the Oliver Endowed Professor of Practice, and the School of Mechanical Engineering, School of Interactive Computing, and School of Psychology. The project is funded by General Motors Corp.
“Our work involves exploring the different types of control schema that will function best as we move toward level three semi-autonomous and level four autonomous cars,” Li said. “Maybe at level three there’s no steering wheel anymore — maybe it’s a joystick — and maybe there’s no speedometer or instrument cluster either, since a semi-autonomous car would automatically obey speed limits.”
The Human-Machine Interface group cooperates with Bruce Walker’s Sonification Lab, sharing assessment tools and expertise. Both labs use miniSim software from the National Advanced Driving Simulator group to help support their testbeds’ simulation and evaluation capabilities.
Li envisions cars with semi-transparent windshields that give a view of the road while also providing important vehicle information along with email, Web pages, and video. If the windshield images suddenly disappear, it could be a signal to the person in the driver’s seat to prepare to take over.
“The automatic transmission, which first appeared about 1950, was a terrible design, and it took decades for it to improve to the point where in the U.S. it has almost full acceptance,” Li said. “Autonomy is going to be the same way — acceptance will be gradual. But 60 years or so from now cars will really drive themselves, and we’ll just lounge around doing what we want.”
Modeling Mixed-Fleet Conflicts
Michael Hunter, an associate professor in Georgia Tech’s School of Civil and Environmental Engineering, uses computer models to study the management and operation of future roadways. His work is looking at a variety of possible traffic scenarios as semi-autonomous and fully autonomous vehicles become a reality.
Hunter directs both the University Transportation Center, a research effort sponsored by the U.S. Department of Transportation that involves Georgia Tech and three partner universities; and the Georgia Transportation Institute, which helps coordinate transportation research with the Georgia Department of Transportation.
His work has thus far turned up concerns that include:
- Disruption and danger: The presence of autonomous cars on roadways could disrupt traffic flow for a number of reasons. Robotic cars, programmed to prioritize safety, would give way to aggressively driven conventional cars. That could become a big problem at rush hour, for example, as drivers entering a highway may find they can cut off the distinctive-looking self-driven vehicles, creating shockwaves that slow or stop traffic. Worse, irresponsible drivers or pedestrians, knowing how autonomous vehicles are programmed, might play dangerous games that involve physically challenging the robots to make them swerve away.
- Surface street bottlenecks: Multiple studies forecast that highly responsive autonomous technology will increase traffic flow on main arteries, with cars and trucks traveling in tight, high-speed formations that respond instantly to changes in speed or conditions. But one result could be gridlock when this greater volume of vehicles hits city streets that still use conventional traffic signals.
- Legislative gridlock: The potential problems between autonomous and driven vehicles could someday spark legislative efforts to ban driven cars, a move that’s sure to be controversial. “I am highly doubtful that any government entities would ban human beings from driving for the foreseeable future,” Hunter said. “This is not going to be a short-term transition. A mixed fleet of human-driven and robotic vehicles — with any number of issues and challenges — is going to be the long-term state of the system.”
Autonomous Racing Test Track
A collaborative research team is using scale-model racing cars to explore methods for keeping an autonomous car under control — or rather to help it keep itself under control.
At the Georgia Tech Autonomous Racing Facility, researchers from the School of Interactive Computing (IC) and the Daniel Guggenheim School of Aerospace Engineering (AE) are racing, sliding, and jumping these one-fifth-scale cars at the equivalent of 90 mph. The goal: to develop maneuvering techniques that can keep an autonomous vehicle on the road and its occupants safe.
James Rehg, a professor in IC, is collaborating on the race track studies with Professor Panagiotis Tsiotras and Assistant Professor Evangelos Theodorou, both of AE. The work is sponsored by the U.S. Army Research Office.
Two cars — custom built by the team — use powerful graphics processing units (GPUs) onboard to supply the computation necessary for autonomous navigation and data capture. Using GPS-based guidance, multiple parallel GPUs employ advanced mathematical techniques to provide the real-time functionality needed for autonomous control at high speeds. Inertial measurement sensors provide additional velocity data.
“Our vehicles are unique because they’re fully autonomous — there’s no tether, no radio connection, and all data processing is done on the vehicle,” said Rehg, who is a member of the Institute for Robotics and Intelligent Machines at Georgia Tech. “We know of studies that are using full-size racing cars and human drivers, but there are some clear advantages to being smaller — we can maneuver our cars very aggressively, and if we crash, no one’s hurt and it’s easy and low-cost to replace the components.”
Computers, he added, can readily convert the information collected from these small cars, so that their performance data becomes comparable to the performance expected from a full-size vehicle.
Designing and building the project’s vehicles was a complex undertaking. Challenges included integrating the onboard computer, sensors, actuators, and software so they work together seamlessly and robustly. The team used gasoline engines at first, but later turned to electric motors that combine small size with plenty of power.
In one series of experiments, the researchers utilized maneuvers used by rally car drivers to control vehicles during a jump. The team was able to program these techniques — which control landings by spinning or slowing a vehicle’s wheels in mid-air — into the guidance systems of its autonomous cars.
Other efforts include vision-based control for autonomous vehicles to augment GPS guidance, and the ability for vehicles to anticipate deadly T-bone collisions before impact and then maneuver automatically to better protect occupants.
“An autonomous vehicle should be able to handle any condition, not just drive on the highway under normal conditions,” said Tsiotras, an expert on the mathematics behind rally car racing. “One of our principal goals is to infuse some of the expert techniques of human drivers into the brains of these autonomous vehicles.”
Exploiting the Electric Grid
The autonomous vehicles of the future may be powered largely by electric engines, which offer energy advantages in stop-and-go urban driving. If so, the presence of millions of high-capacity car batteries could have major implications for the U.S. electric grid.
Valerie Thomas, who researches renewable energy, is studying the interplay between electric vehicles and the grid. She notes that plugging in a host of electric vehicles could increase the U.S. system’s flexibility. Electric vehicles are often plugged in when not in use, allowing them to charge with under-utilized power — late at night as demand goes down, or when wind power is high, or on sunny days with high solar power.
By using the excess power of the grid, the presence of myriad electric vehicles could result in lower electricity costs, said Thomas, who is Anderson Interface Professor of Natural Systems in Georgia Tech’s H. Milton Stewart School of Industrial & Systems Engineering and has a joint appointment in Georgia Tech’s School of Public Policy.
“Some have argued that electric vehicles’ flexibility means they’ll be consuming mostly ‘dirty’ power such as low-cost coal-generated power,” Thomas said. “But our studies have found the opposite — these vehicles’ flexibility lets them take advantage of renewable electricity when it is available, and to a large degree can solve the problem of the intermittency of wind and solar.”
And if charged car batteries could send some electricity back into the grid when required, smoothing out power crunches, there could be additional cost savings for all electricity users.
Improving Driver Assistance
Many driver-controlled cars are now equipped with sensors — including cameras, radar, and laser proximity devices — that detect nearby vehicles and other aspects of the environment. These level-two driver-assistance systems log vast quantities of data on active cruise control, automatic braking, lane changes, and other performance elements.
Byron Boots, an assistant professor in Georgia Tech’s School of Interactive Computing, is performing statistical analysis on sensor data from a large fleet of level-two vehicles with driver-assistance capabilities. Working with sponsor BMW AG, Boots and his team are investigating the cars’ ability to predict when a potentially hazardous event is imminent and then effectively communicate the situation.
Driver assistance, Boots said, is getting a lot of attention from many car makers. Companies are exploring the theory that advising the driver of impending danger could result in a better driving experience than having control suddenly — and alarmingly — wrested away.
“For example, you may be entering a near-miss situation where another vehicle is about to merge into your lane, cutting you off,” Boots said. “If your car can perceive the other vehicles on the road and use machine learning to predict the dangerous situation before it happens, it could instantly advise the driver of the hazard rather than just taking over control. That approach may not just help to make transportation safer, but also help convince people that automation really is able to protect them.”
Briefing the Government
The advent of autonomous transport has the attention of government. Sebastian Pokutta, who is David M. McKenney Family Assistant Professor in Georgia Tech’s School of Industrial & Systems Engineering, recently co-authored a white paper for the Computing Community Consortium that was presented to White House representatives and others at a National Science Foundation workshop.
“Many automakers believe that by 2020 we can have fully functional autonomous vehicles on the road,” Pokutta said. “But for that to happen, there have to be legislative and policy decisions that address a number of critical technology, infrastructure, and other issues.”
Some of the key points in Pokutta’s analysis include:
- High-speed platooning of cars and trucks could translate into faster commute times. That in turn could add to urban sprawl as people move farther out into the hinterlands. Moreover, car sharing could mean that vehicles will be continually in use, lowering demand for parking and potentially changing urban land use.
- Self-driving prototypes rely heavily on special physical and mapping infrastructure. Extensive investment will be required to bring those kinds of infrastructure changes to the whole nation or to find other ways to overcome these limitations. Even then, self-driving cars may lose their bearings in unexpected situations such as construction, detours, road closures, or unusual weather conditions.
- The GPS-dependent mapping systems that guide autonomous vehicles must be made 100 percent reliable. If signals cut out in bad weather, under bridges, or inside buildings and tunnels, serious problems could ensue.
“One huge benefit from self-driving vehicles could be a major reduction in traffic accidents. Every year 40,000 people die on U.S. roads, and drunk driving results in an estimated $200 billion in costs,” Pokutta said. “Autonomous systems taking control away from impaired or deranged operators — in trains and aircraft as well as motor vehicles — could save many lives and a great deal of money.”
Planning for Freight-Vehicle Autonomy
Increasing automation won’t affect only cars. Freight vehicles of every type, from tractor-trailers to delivery trucks, will also be changed by autonomy in ways that aren’t yet known.
A team led by Catherine Ross, Harry West Professor in Georgia Tech’s School of City and Regional Planning (SCaRP), and Tim Welch, an assistant professor in SCaRP, is investigating what may happen as freight vehicles adopt technology that lets them communicate with one another. They could start to drive in tightly packed groups — including high-speed convoys on interstate roads — and follow more efficient routes based on real-time road conditions.
“Eventually the entire human aspect of many freight deliveries — or at least the travel part of them — could become autonomous,” Welch said. “We’re studying what that development may look like in the next five, 10, or 15 years.”
Planners will have to develop new approaches to make current road systems more adaptable to autonomous traffic movement. And policies and legislation will be needed to enable the construction of new highway and street infrastructure to accommodate the coming changes over the long term.
The team is working under a grant from the Transportation Research Board, which is part of the National Academies of Science under the National Cooperative Highway Research Program. The project, a joint effort with Booz Allen Hamilton Inc., is a national research effort.
“The time horizon for a fully autonomous fleet of any vehicles, passenger or freight, is going to be a pretty long one,” Welch said. “Managing that entire process over the long term is going to be a challenging task.”
Anticipating Market Changes
Vivek Ghosal believes that once the autonomous revolution becomes established, the economic world that underpins wheeled transport will never be the same. Ownership, liability, and manufacturing paradigms could all change in major ways.
And, like others, he believes the conversion process will be costly and that both government and industry will have to pick up a very large bill. Whether cars are semi-autonomous or fully autonomous, electric or fuel cell powered, they’ll require wide-scale improvements in roads, traffic signals and controls, road markings, and signage. They’ll also need a vast charging/fueling infrastructure that today barely exists.
“Even if you could let level-four autonomous cars out on the streets right now, there would be serious problems,” said Ghosal, an auto industry specialist who is Richard and Mary Inman Professor in Georgia Tech’s School of Economics. “Major infrastructure investments are needed to operationalize this technology.”
Observations from Ghosal’s research include:
- Ownership of cars could be substantially reduced under an autonomy-centered paradigm. Even today, car-sharing markets in Europe are expanding quickly, and major car makers there are scrambling to compete with Car2Go, Zipcar, and others. The combination of quickly available level-four cars and extensive car sharing will likely produce lower demand for personally owned automobiles.
- Changes in ownership patterns will likely propel current automotive, lending, and insurance markets into unmapped territory, as consumers gradually learn to regard cars as a transportation service rather than a purchase.
- Self-driving vehicles are essentially information technology-enabled devices, a fact that software companies realize. Ghosal believes the autonomous prototypes being developed by companies like Google and Apple aren’t aimed at starting new car manufacturing corporations but are instead focused on developing definitive operating systems for autonomous control. This proprietary software could become a costly necessity for established automotive manufacturers as they evolve driverless vehicles.
Said Ghosal: “This is perhaps the most significant disruption in this industry since the invention of the assembly line by Ford.”
One thing is certain: The self-driving revolution is on its way. What isn’t known is what form it will take as it becomes a reality. Georgia Tech research teams will continue to study and develop effective real-world approaches as the transformation continues.
Rick Robinson is a science and technology writer in Georgia Tech’s Institute Communications. He has been writing about defense, electronics, and other technology for more than 20 years.