Not to pick on Tesla – Many companies do this – But it really needs to be more rigorous in discussing its numbers. Take the most recent press release on its car safety as an example. In its Q3 2018 Vehicle Safety Report, it wrote the following:
“
- Over the past quarter, we’ve registered one accident or crash-like event for every 3.34 million miles driven in which drivers had Autopilot engaged.
- For those driving without Autopilot, we registered one accident or crash-like event for every 1.92 million miles driven. By comparison, the National Highway Traffic Safety Administration’s (NHTSA) most recent data shows that in the United States, there is an automobile crash every 492,000 miles. While NHTSA’s data includes accidents that have occurred, our records include accidents as well as near misses (what we are calling crash-like events).
“
While the numbers look impressive, they are not enough to prove that Tesla’s cars are statistically safer than other cars. Anyone who took any decent statistics course will point out that in order to assert that Tesla’s cars are statistically different in terms of miles-per-crash, the company not only need to disclose averages, but also their variabilities (or technically called standard deviations) in miles-per-crash.
The reason is the following. Some people have a crash after driving very short distances while others after driving very long distances. The NHTSA’s average of 492,000 miles are the average among all those accidents. The same for Tesla, some drivers have a crash after driving short distances, while others after long distances. The concern is that if the variance among those Tesla drivers are much much wider than that among all drivers in the NHTSA data, then statistically one cannot say that Tesla cars are safer.
Having said that, I think chances are that Tesla cars indeed have fewer accidents on a per mile basis. However, we still need numbers to back up. In this case, we need to know the statistical distributions of miles-per-crash for Tesla drivers as well as for all the other drivers.