Vision-Based Framework for Automated Testing of Automotive HMI Systems Using Deep Learning Techniques AVL Technical Centre, Pvt., Limited

Format:

Book

Conference/Event

Author/Creator:

Lad, Rakesh Pramod, author.

Contributor:

Mehrotra, Soumya

Mishra, Arvind

Conference Name:

Symposium on International Automotive Technology (2026) (2026-01-28 : Pune, India)

Language:

English

Physical Description:

1 online resource cm

Place of Publication:

Warrendale, PA SAE International 2026

Summary:

Nowadays, digital instrument clusters and modern infotainment systems are crucial parts of cars that improve the user experience and offer vital information. It is essential to guarantee the quality and dependability of these systems, particularly in light of safety regulations such as ISO 26262. Nevertheless, current testing approaches frequently depend on manual labor, which is laborious, prone to mistakes, and challenging to scale, particularly in agile development settings. This study presents a two-phase framework that uses machine learning (ML), computer vision (CV), and image processing techniques to automate the testing of infotainment and digital cluster systems.The NVIDIA Jetson Orin Nano Developer Kit and high-resolution cameras are used in Phase 1's open loop testing setup to record visual data from infotainment and instrument cluster displays. Without requiring input from the system being tested, this phase concentrates on both static and dynamic user interface analysis, including screen transitions, animations, and error messages. Among the methods used are optical character recognition (OCR) for on-screen text validation, convolutional neural networks (CNNs) for screen classification, and object detection for user interface verification. Automated anomaly detection and interface behavior evaluation are made easier with this method.Phase 2 suggests integrating a Hardware-in-the-Loop (HIL) simulator to transform the system into a closed-loop testing environment. The vision-based system will assess system responsiveness and end-to-end behavior, while the HIL setup will produce simulated user inputs and vehicle network data (such as CAN, Ethernet). This thorough framework tackles important issues like complex system integration, multimodal interaction testing, and managing cognitive load. In order to support the creation of safer, more user-friendly infotainment and digital cluster systems that are in line with Advanced Driver Assistance Systems (ADAS) standards, it seeks to decrease the amount of manual testing effort, increase test coverage, and improve consistency

Notes:

Vendor supplied data

Publisher Number:

2026-26-0571

Access Restriction:

Restricted for use by site license