China releases first autonomous vehicle road testing report

Beijing recently issued the "Beijing Autonomous Vehicle Road Testing Report 2018"; the first report on autonomous vehicle road testing in China. The Report releases information and data derived from road tests by 56 cars from eight companies (Baidu, NIO, BAIC BJEV, Daimler, Pony.ai, Tencent, Audi, and Didi Chuxing) in both enclosed areas and open roads in Beijing during 2018.

In this short blog post we consider the information that is, and is not, included in the Report.

• Beijing keeps expending its testing areas

According to the Report (available here), Beijing has opened 44 roads in 4 districts with a total distance of 123 km for autonomous vehicle testing since February 2018 and is the leading city in China. Beijing has built and opened the first V2X reversible testing road with a distance of 12 km, combining traffic light systems, intellectual sensor systems and connected autonomous vehicles to provide an integrated testing environment. And it is planned that autonomous vehicle testing areas in Beijing will expand to 500 km², which includes open roads covering more than 2,000 km in 2022.

• Baidu tops industry peers in road testing

In the Report, which includes results from 8 companies, Baidu tops industry peers with the most test license plates (45), the most test vehicles on road (45), the longest distance driven (140,000 km), and the most diverse test scenarios.

• No disengagement data

The Report includes two categories of test data: the number of test cars and the distance travelled by those cars.  Significantly, no data is provided regarding disengagement. 'Disengagement' includes situations in which a self-driving vehicle's systems are unable to process current conditions, forcing it to pass control back to the human driver, and also where a human manually retakes the wheel or overrides a car's decision also count as disengagements.

This decision is interesting for two reasons.  First, this data is available; all disengagement data is required to be reported to an authorized third-party institution (i.e. the Beijing Innovation Centre for Intelligent Mobility) on a monthly basis.

Second, the absence of this data means that it is not possible to calculate the "miles per disengagement" (MPD) achieved by each company (ie. the distance travelled by the automated system before human intervention was required).  This metric is considered by many commentators to be a more measure of safety than distance travelled alone.  Notably, disengagement data is included in the annual reports published by the Californian Department for Motor Vehicles.

It will be interesting to see whether this data is included in subsequent reports.

• Categories of disengagement

Although no technical data is provided regarding disengagement, the Report does identify four categories of disengagement that have occurred during the testing:

• system failure caused by sensor failure, map loading anomaly, positioning deviation, system delay anomaly, data logging device failure;
• strategic deviancies caused by obstacle identification errors, social vehicle behavior prediction errors, path planning errors, vehicle stagnation;
• expected take-over caused by vehicles illegally occupying lanes, non-motorized roads and construction; and
• manual take-over caused by engineers changing equipment, engineers re-calculating path.

This Report provides a unique insight into the testing of autonomous vehicles in Beijing since testing regulations were first published in December 2017.  While it is regrettable that not all available information is included, the Report nevertheless provides interesting information relating to the testing and evaluation equipment scenarios, roads, and more importantly the problems encountered by the testing companies, including reasons for disengagement.  We would look forward to seeing more reports being released by other cities in China where testing is taking place.