Cruise publishes a baseline for their safety analysis

Summary: a Cruise study suggests they are better than a young male ride hail driver in a leased vehicle. However, this result is an estimate, because there is not yet enough data to have a firm conclusion.

I am glad to see Cruise release a paper describing the methodology for computing the human driver baseline, which they had not previously done. Same too for their “meaningful risk of injury” estimation method. And it is good to see a benchmark that is specific to a deployment rather than a US average.

Cruise has published a baseline study for their safety analysis here:
blog post: https://getcruise.com/news/blog/2023/human-ridehail-crash-rate-benchmark/
baseline study: https://deepblue.lib.umich.edu/handle/2027.42/178179
(note that the baseline study is a white paper and not a peer reviewed publication)

The important take-aways from this in terms of their robotaxi safety analysis are:
The baseline is leased ride hail vehicles, not ordinary privately owned vehiclesThe drivers of the baseline are young males (almost a third are below 30 years old)A “meaningful risk of injury” threshold is defined, but somewhat
arbitrary. They apparently do not have enough data to measure actual
injury rates with statistical confidence. Given that we have seen two
injuries to Cruise passengers so far (and at least one other injury
crash), this is not a hypothetical concern.

It should be no surprise if young males driving leased vehicles as
Uber/Lyft drivers have a higher crash rate than other vehicles. That is
their baseline comparison. In fairness, if their business model is to
put all the Uber and Lyft drivers out of work, perhaps that is a useful
baseline. But it does not scale to the general driving population.

See also When Is a Recall Not a Recall?

A conclusion that a Cruise robotaxi is safer (fewer injuries/fatalities)
than an ordinary human driver is not quite supported by this study.

It is not an “average” human driver unless you only care about Uber/Lyft. If that is the concern, then OK, yes, that is a reasonable comparison baseline.I did not see control for weather, time of day, congestion, and other conditions in the baseline. Road type and geo-fence were the aspects of ODD being used.There is insufficient data to have a conclusion about injury rates, although that will come somewhat soonWe are a long way away from insight into how fatality rates will turn
out, since the study and Cruise have about 5 million miles and San
Francisco fatality rate is more like one per 100 million milesThe Cruise emphasis on “at fault” crashes in other messaging is a
distraction from crash outcomes that must necessarily include the
contribution of defensive driving behavior (avoiding not-at-fault
crashes)

This study could support a Cruise statement that they are on track to being safe according to their selected criteria. But we still
don’t know how that will turn out. This is not the same as a claim of proven
safety in terms of harm reduction.

A different report does not build a model and estimate, but rather
compares actual crash reports for robotaxis with crash reports for ride
hail cars and comes to the conclusion that Cruise and Waymo operated at 4
to 8 times as many crashes as average US drivers, but that their crash
rate is comparable to ride hail vehicles in California.

https://www.researchgate.net/publication/373698259_Assessing_Readiness_of_Self-Driving_Vehicles