It Takes An Army Of Engineers To Make Self-Driving Cars Actually Self-Drive

The race to create a real self-driving car is heating up, with startups like Waymo deploying its autonomous taxis across U.S. cities, billions of dollars being plowed into new technology and Tesla shouting that its cars will one day really be able to drive themselves. Now, a new report has looked into what it really takes to run an autonomous taxi service, and it includes a fleet of engineers hundreds of miles away that can control the cars in some situations.

People Are Relying on ADAS to Do Things it Can’t Do

Self-driving cars have been in service across places like San Francisco, California, and Phoenix, Arizona, for a few years now, offering rides around town without a pesky driver to chat with while you get around. For the most part, they’re guided around by a vast array of sensors and cameras onboard that survey the surroundings and work with onboard computers to calculate the best way to act.

However, a new report from the New York Times has revealed that this isn’t the only way autonomous vehicles travel around. In fact, self-driving car companies have an army of engineers on hand to help their cars out whenever they need it:

Inside companies like Zoox, this kind of human assistance is taken for granted. Outside such companies, few realize that autonomous vehicles are not completely autonomous.

For years, companies avoided mentioning the remote assistance provided to their self-driving cars. The illusion of complete autonomy helped to draw attention to their technology and encourage venture capitalists to invest the billions of dollars needed to build increasingly effective autonomous vehicles.

“There is a ‘Wizard of Oz’ flavor to this,” said Gary Marcus, an entrepreneur and a professor emeritus of psychology and neural science at New York University who specializes in A.I. and autonomous machines.

These engineers aren’t sat watching a constant stream from every self-driving car operating in the U.S. Instead, they’re called on whenever the autonomous vehicles reach an obstacle or an incident that they don’t know how to respond to, reports the NYT.

When that happens, the video feed flashes up for an engineer to see, as well as input from all the other sensors dotted around an autonomous vehicle. They can then offer the vehicle a route through or out of the situation it finds itself stuck in, as the NYT adds:

If a Zoox robot taxi encounters a construction zone it has not seen before, for instance, a technician in the command center will receive an alert — a short message in a small, colored window on the side of the technician’s computer screen. Then, using the computer mouse to draw a line across the screen, the technician can send the car a new route to follow around the construction zone.

“We are not in full control of the vehicle,” said Marc Jennings, 35, a Zoox remote technician. “We are providing guidance.”

This all makes it sound like the human input is a last resort for companies like Zoox, however autonomous car startups turn to their human overlords for all kinds of issues with the self-driving vehicles.

In the NYT report, examples include a slightly different roadworks zone than the autonomous cars were programmed on before. In another instance, human input was required when a self-driving car was directed around a parked fire truck. Both of these are situations that we might take a moment to consider before driving on as we’ve done hundreds of times before.

As such, it might not be surprising to learn that autonomous taxi firms like Zoox and Waymo wouldn’t disclose how many engineers they employ to teach self-driving cars to respond to unknown situations. They also didn’t confirm to NYT how often people are called in to help out autonomous taxis.

That begs the question, then, surely it would be cheaper just to employ a driver to sit in the car and steer it in the correct, safe, legal manner?