Testing Solutions for Autonomous Driving

Autonomous driving increases the requirements on testing systems. WKS Informatik developed an integrated solution that can be used for very first prototypes as well as for the end-of-line testing.

DRIVE SAFELY
There are many benefits which autonomous cars would bring into everyday life. According to McKinsey estimations [1], autonomous vehicles in a widespread use, would eliminate up to 90 % of crashes, thus reducing not only the number of human victims, which is one of the main goals of autonomous driving, but also saving enormous costs on health and vehicle insurance. Moreover, up to 50 min of commuting time could be freed up, parking space could be reduce by billions of square metres and many disabled persons would be facilitated with increased mobility.

In 2016, then-president Obama published an article explaining the US government support for autonomous driving, stating that: “Safer, more accessible driving. Less congested, less polluted roads. That’s what harnessing technology for good can look like. But we have to get it right [2].” In order to get it right, several technical concerns about the safety of driverless cars need to be no longer an issue. The US Department of Transportation released in 2016 a 15 point checklist to serve as guideline for self-driving car manufacturers. In addition to ethical and law issues, one of the 15 points reffers to one of the most critical aspects car manufacturers face, validation: “Automakers need to develop testing and validation methods that account for the wide range of technologies used in driverless cars. Their tests should include simulation, test track and on-road testing.” Software and hardware reliability is one of the most important issue OEMs struggle with.

Why? The SAE defined 6 levels of autonomy, FIGURE 1, [3]. Levels 0 and 1 are obviously routine today. For the level 2, the driver must be prepared to intervene immediately in case the automated system is unable or not allowed to handle the situation on its own. But studies have revealed that when a person is disengaged from driving it takes more than 10 s to fully reengage and take action [4]. Critically, for an emergency taking place at 100 km/h, 10 s mean more than 270 m, during which the vehicle might collide with the traffic lane boundary or other road users!

BIG DATA AND THE CONNECTED WORLD
Another key point in the autonomous driving world is the connectivity between cars. In order to function autonomously, vehicles need to sense the world as much as human drivers do today. Accurate positioning is critical, meaning that today’s accuracy, mostly relying on radar, lidar, camera, GNSS and ultrasound sensors, must be increased to a matter of centimetres and that the positioning information must be exchanged quickly at all time between the autonomous vehicles, in order to ensure safety not just for the vehicles, but also for the pedestrians in traffic. While enabling the passengers with more free time, autonomous driving must also make room in its data transfer for passenger entertainment.

Every OEM works on connected platforms, in various co-operations. For faster developments an open-platform would be needed, allowing easier maintainance and higher standardisation. In the same manner, testing systems should allow the same openess an connectivity. This not just allows extensive reusability of existing resources, but also enables testing platforms to integrate new technologies sooner and with lower costs.
All this intensive data exchange between cars also means that enormous amounts of data will be transmitted. Not only that, but also stored, analysed and managed. Barclays analysts estimate that “a single autonomous car, with all of its sensors, cameras, and lidar, could generate as much as 100 GByte/s” [5]. So testing systems nowadays need to address a new issue: Big (Analogue) Data or how to deal with these amounts [6].

TESTING FOR AUTOMATED DRIVING
The above can be summarised into five key points for testing:
– Validation is the most critical aspect in quality assurance, covering various tests, such as regression, performance, functional and security tests, in order to insure reliable product functionality.
– Big Data must be analysed and managed in a smart way, in order to avoid so-called data garbage and reduce analysis times.
– Cloud solutions for data communication must be enabled through the testing systems.
– Sensor accuracy is top criteria for every autonomous vehicle.
– Testing systems should be based on open, flexible platforms, allowing testing of various sensor types and the integration of various simulation scenarios.

If you try a comparison of different test levels from the subfields of system testing, design verification, product validation, end-of-line testing and prototype testing, there are, from the author’s point of view, similar issues and therefore also similar solutions.

OPEN TESTING PLATFORM
Using National Instruments technology, WKS Informatik GmbH developed the RTStand platform for testing integrated ECUs throughout all test levels from prototyping to end-of-line tests. RTStand combines the functionality of a traditional HiL system with those of a real-time automated test equipment (ATE) and uses methods of data reduction, for example for the minimisation of log data. In addition to that, RTStand provides a powerful technique of data navigation, to examine relations between test result, test report and actual values measured in timing diagrams – with the emphasis on finding relations, instead of only searching.

THE NEED FOR ADAPTIVE LOGGING
In order to insure components reliability, OEMs have to fulfil various norms. The LV 124 (and its related norms VW 80000,ISO 16750-2 etc.) is one of the most important norms for automotive validation. Developed by the German OEMs, it describes various testing scenarios for car components and the way they must behave under specific circumstances.
Having the purpose of detecting even the slightest failures in a signal and requiering a sampling rate of 100 kHz for all tested signals, the LV 124 produces huge amounts of data that need to be analysed. Assuming that such big testing resources are available, finding the one short sporadic error of 1 ms will be a challenge.

This idea drove WKS Informatik to develop an adaptive logging process for analysing data on-the-fly and thus reducing the amount of data produced. This not only saves IT resources, but also enables a much faster data analysis, through much easier data navigation.
As shown in FIGURE 2, it is possible to determine expected values like limits or envelope curves for the signals to be tested. These curves are used at runtime in order to determine whether a signal behaves as expected under given conditions. In a good case, the signal will be logged with a reduced rate – the only interesting information here being the approximate signal trend-. In a failedcase, the signal will be logged with the highest rate (for example 100 kHz).

Let us take an example where 15 min of usual logging produce 71 GByte of data. Using adaptive logging, which will only use high-sampling during the 1 ms event and for example 1 Hz sampling at the rest of the time, only 10 kByte of data will be produced, reducing the amount of data by seven million times.

BIG DATA ANALYSIS
By combining this procedure and log viewing capabilites, a quick and simple navigation of the test data is ensured. RTStand integrates all this natively and through complementary products for validation: Tube Analyzer – as the measurement and analysis unit – and RTStand LV 124 – as a fully-automated testing solution.

INTEGRATION OF CLOUD SOLUTIONS
The connected car needs intensive data communication, as explained above. RTStand integrates plugins for various Industry 4.0 needs, such as data saving into and visualising from cloud technologies.
Comparing the efficiency between different test benches can be an important aspect while testing various car components. Each test bench can correspond to a specific component and through cloud communication their interaction can be observed and improved.

RADAR ACCURACY
As much as we as humans rely on our senses to navigate through the daily city jungle, so do autonomous vehicles rely on the accuracy of their sensors. A vehicle must be able to react at speeds of even over 150 km/h. So vehicle sensors must work more accurately than human senses.

RTStand integrates radar target simulation solutions for both 24- and 77-GHz radars, characterisation and chirp analysis tools for radar, which are used extensively in hardware development and end-of-line testing,FIGURE 3.

ONE PLATFORM TO RULE THEM ALL
WKS Informatik integrates all the above solutions and concepts into its RTStand platform, providing one solution for testing from early prototyping, until the end-of-line. Because as an OEM you have the certainty that at all levels in your product development cycle, the same technology and the same reliability of testing results is guaranteed.
An open platform can bring both kind of manufacturers – automotive and non-automotive – on the same testing grounds, allowing them to concentrate less on interfaces and more on quality.

SUMMARY
Autonomous driving is happening. Traditional manufacturers must step up to the challenges from the newest players on the market, such as Google, Tesla, even Apple and Amazon, and make the transition to being a mobility service provider in the connected world of the Internet of Things.

While producing a reliable and client-appealing car, OEMs must be sure that their products function safely under all circumstances. Recently Tesla announced that they will skip testing steps in order to boost production and reach deadlines [7], by ordering ready-to-use equipment. If this becomes a trend, there will be more pressure on the automotive suppliers to validate thoroughly and rely on better risk management strategies.