About half of traffic fatalities worldwide are vulnerable road users (VRUs) such as pedestrians or cyclists. VRUs are always the weaker collision partner in road traffic in the event of a collision with a vehicle. For the realization of safe autonomous driving, the safe detection and classification of VRUs and the prevention of accidents is essential. VRUs usually possess a high degree of freedom in terms of their movement capabilities, which allows them to suddenly change their movement behavior. This poses a great challenge to safety systems in terms of situation detection and interpretation of situation criticality assessment. Consequently, the improvement of safety systems that specifically detect and analyze the movement behavior of pedestrians is an essential part of current developments and Euro NCAP test scenarios. C-ISAFE is therefore researching algorithms to extract the behavior of VRUs from sensor signals and to be able to recreate it with specially developed VRU dummies.
Pedestrian detection
Pedestrians have the ability to quickly and unpredictably change direction and speed and suddenly step onto the road. Children up to elementary school age in particular have a high potential for danger. In addition to dangerous behavior, such as playing at the side of the road or not knowing the traffic rules, physical development that is not yet complete also plays a decisive role. For example, they have trouble assessing approaching vehicles. In addition, due to their small body size, they are almost undetectable to the sensor system behind vehicles and are thus detected even later when stepping out from between vehicles.
Therefore, it is of great significance for the implementation of automated driving in urban scenarios to be able to reliably detect pedestrians regardless of their age, to understand the situation and to be able to interpret intentions. In C-ISAFE, radar- and camera-based methods are investigated and developed to realize robust and fast detection of pedestrians. High resolution sensors for the near field are used. By evaluating the micro-Doppler effect (velocity distribution due to movements of the extremities) as well as high-resolution distance information, important insights into the pedestrian's behavior can be gained in addition to enhanced detection. For this purpose, C-ISAFE has developed patented methods to interpret the micro-Doppler profile of VRU into the algorithm for detection and behavioral intention. Early indicators are being investigated to register a change in motion (e.g., a change in direction) in a timely manner. Based on this, methods are developed to include these in a path prediction. So far, such path predictions are mainly based on only one environment sensor. To achieve the best possible pedestrian detection, C-ISAFE is investigating methods to combine complementary sensors such as camera and radar. By fusion of both sensors with the help of AI methods, their advantages shall be used and a high robustness against bad weather conditions like rain or fog shall be achieved.
Dummies
The development of safe vehicles and new driving functions requires a targeted recreation of critical and dangerous traffic situations. A dummy pedestrian imitates not only the look of a human but also his or her movements and enables the safe recreation of critical traffic situations. Due to the progressing developments in the field of automated driving, starting from the sensor hardware via the recognition and interpretation algorithms up to the driving function, the requirements for a dummy are increasing. The challenge here is to reproduce the characteristic features and movements of pedestrians from the perspective of the automotive environment sensors by a test system close to reality. The development of such dummies and associated test methods is a research focus of C-ISAFE. In order to be able to simulate different walking styles and motion sequences more realistically than already available dummies, a new generation of pedestrian dummies (ArChi - Artificial Child and PAdDy - Pedestrian Adult Dummy) is being developed at the institute. These are driven by pneumatic muscles (comparable to human muscles) and enable realistic movement behavior. In addition to leg movements, arm and head movements are also simulated. Metallic components are completely omitted to enable a realistic radar signature and to reduce damage in the event of possible collisions with test vehicles. Various solutions are available for moving the dummy pedestrians in the indoor test facility, enabling different complex test scenarios.
VRU-behavior analysis
C-ISAFE has built a pedestrian simulator for a more detailed analysis of pedestrian behavior. With this simulator it is possible to analyze the behavior and typical movement patterns of pedestrians in road traffic and to integrate it into the development of systems for integral safety and test methods. A head-mounted display (HMD) and a motion capture system are used for this purpose. The movements of the test person can thus be transferred to a simulation environment, which the test person sees through the HMD. By viewing the own (simulated and) moving body, the highest possible degree of reality is achieved.
Investigations range from the behavior while crossing a road and the analysis of the so-called gap acceptance to the body posture while crossing to optimize simulation methods for injury mechanisms. In particular, the movement behavior of children in different scenarios when crossing the roadway was recorded and analyzed. This motion analysis forms the scientifically based development foundation for the child dummy developed at C-ISAFE.
Examples of use
- Studies on pedestrian detection from environmental sensors
- Driving and evasion tests with cyclists
- Investigation of object detection with environmental sensors up to the crash point
Publications
- P. Held, D. Steinhauser, A. Koch, T. Brandmeier, and U. T. Schwarz, "A Novel Approach for Model-Based Pedestrian Tracking Using Automotive Radar," in IEEE Transactions on Intelligent Transportation Systems, vol. 23, no. 7, pp. 7082-7095, 2022, doi: 10.1109/TITS.2021.3066680.
- F. Sezgin, D. Vriesman, P. Held, A. Zimmer, and T. Brandmeier, "A Deep Learning Approach for Pedestrian Behavior Interpretation Based on Radar Point Clouds," 2021 18th European Radar Conference (EuRAD), 2022, pp. 66-69, doi: 10.23919/EuRAD50154.2022.9784515.
- D. Steinhauser, P. Held, B. Thöresz, and T. Brandmeier, "Towards Safe Autonomous Driving: Challenges of Pedestrian Detection in Rain with Automotive Radar," 2020 17th European Radar Conference (EuRAD), Utrecht, Netherlands, 2021, pp. 409-412, doi: 10.1109/EuRAD48048.2021.00110.
- P. Held, D. Steinhauser, A. Kamann, A. Koch, T. Brandmeier, and U. T. Schwarz, "Micro-Doppler Extraction of Bicycle Pedaling Movements Using Automotive Radar," 2019 IEEE Intelligent Vehicles Symposium (IV), Paris, France, 2019, pp. 744-749, doi: 10.1109/IVS.2019.8813828.
- P. Held, D. Steinhauser, A. Kamann, A. Koch, T. Brandmeier, and U. T. Schwarz, "Normalization of Micro-Doppler Spectra for Cyclists Using High-Resolution Projection Technique," 2019 IEEE International Conference on Vehicular Electronics and Safety (ICVES), Cairo, Egypt, 2019, pp. 1-6, doi: 10.1109/ICVES.2019.8906495.
- D. Steinhauser, P. HeId, A. Kamann, A. Koch, and T. Brandmeier, "Micro-Doppler Extraction of Pedestrian Limbs for High Resolution Automotive Radar," 2019 IEEE Intelligent Vehicles Symposium (IV), Paris, France, 2019, pp. 764-769, doi: 10.1109/IVS.2019.8813850.
- P. Held, D. Steinhauser, A. Kamann, T. Holdgrun, I. Doric, A. Koch, and T. Brandmeier, "Radar-Based Analysis of Pedestrian Micro-Doppler Signatures Using Motion Capture Sensors," in 2018 IEEE Intelligent Vehicles Symposium (IV), Changshu, China, 2018, pp. 787-793, doi: 10.1109/IVS.2018.8500656.
- T. Holdgrün, I. Doric, T. Brandmeier, T. Fuchs, J. Mihlbauer, P. Steinert, S. Peldschus, "A Virtual Reality based Approach for Researching Pedestrian to Vehicle Collisions," in 2018 IEEE Intelligent Vehicles Symposium (IV), Changshu, China, 2018, pp. 1318-1325, doi: 10.1109/IVS.2018.8500524.
Contact
Prof. Dr.-Ing. Thomas Brandmeier
Phone: +49 841 9348-3840
Room: H023
E-Mail: Thomas.Brandmeier@thi.de