Designing Effective HMIs for Automated Vehicles
As vehicles become increasingly automated, the role of the human changes from driver to supervisor and, at higher levels of automation, to passenger. This transition also changes the role of the human–machine interface (HMI). Rather than simply displaying vehicle information, the HMI becomes an important communication channel between the automated driving system and the people inside the vehicle.
Research conducted by University of Ljubljana (UL) within the FRODDO project examines how these interfaces should be designed, developed and evaluated to support understandable, predictable and comfortable automated driving.
Designing the right information for the right situation
There is no single HMI solution suitable for all levels of automation. In SAE Level 3 driving, interfaces must primarily support safe transitions of control when the driver is required to resume the driving task. At Level 4, the focus increasingly shifts towards communicating system status, Operational Design Domain (ODD) information and relevant vehicle actions. At Level 5, where occupants are passengers rather than drivers, the main challenge becomes providing enough information to understand and anticipate vehicle behavior without unnecessarily demanding their attention.
This means that automated vehicles should not simply display everything the system knows. During routine operation, minimal or on-demand information may be sufficient. Before a relevant maneuver, a short anticipatory message can explain what the vehicle is about to do. When an unexpected or potentially concerning event occurs, additional information about the event, reason for the vehicle's response and expected outcome can help passengers understand the situation. The explicitness of HMI communication should therefore increase with passenger relevance and uncertainty, rather than with the technical complexity of the automated driving system.
From concept to multimodal HMI
Information can be communicated through visual, auditory and tactile interfaces. Visual displays are suitable for persistent information such as route progress and vehicle status, while auditory messages can attract attention without requiring passengers to look at a display. Tactile cues can provide discreet notifications or reinforce information communicated through another modality.
The challenge is not to provide as much information or use as many modalities as possible. Instead, HMI development should determine what information is needed, when it should be communicated, and which modality or combination of modalities is most appropriate.
Within the FRODDO Ljubljana pilot, UL is applying this approach to the development of multimodal interfaces combining visual information, spoken notifications and tactile cues. Different scenarios allow these concepts to be investigated under routine as well as more challenging automated-driving conditions.
Evaluating more than usability
HMI development should be iterative: design, prototype, test, evaluate and redesign. Driving simulators provide an important intermediate step because they allow interfaces to be evaluated under controlled and repeatable conditions before transferring them to real vehicles.
Figure 1: Simulator set-up used at University of Ljubljana, Faculty of Electrical Engineering
Evaluation should also go beyond asking whether users liked an interface. Questionnaires can assess usability, comprehension, predictability, comfort, trust, acceptance and perceived safety. However, these subjective measures can be complemented by behavioral and physiological measurements. UL overcomes this barrier by combining self-reported measures with a sensor fusion of eye tracking, pupilometry, electrodermal activity, cardiovascular measures and other physiological and behavioral signals to better understand the user and their current state. By synchronizing these measurements with vehicle behavior and HMI events, researchers can investigate what happens when, for example, the vehicle changes lane, encounters a hazard or communicates an upcoming maneuver. This event-based evaluation can reveal effects that may not be apparent from an overall questionnaire completed after the drive.
Moreover, use of wearables can provide us with a direct view of the participants’ experiences. For example, using an eyetracker provides us a participant’s-eye view offering a direct impression of the automated driving scenario, involvement with a non-driving related task, interaction with the HMI and the participant’s visual experience.
Such recordings can be further used for analysis of visual attention during the studies, showing detected objects through bounding boxes together with the participant’s gaze position. These recordings illustrate how behavioural data can complement physiological signals and questionnaires, helping researchers understand not only how users evaluate an HMI, but also where they direct their attention and how they visually interact with the information presented during automated driving.
Ultimately, there is no single measure of how good an HMI is. An interface must balance transparency with information overload, predictability with unnecessary interruption, and perceived safety and trust with appropriate understanding of the automated system. The goal is therefore not to show passengers everything an automated vehicle knows. It is to provide the right information, through the right channel, at the right moment. Through iterative development and multimodal evaluation, FRODDO is working towards HMIs that make future automated mobility more understandable, predictable and comfortable for its users.
Authors: Kristina Stojmenova Pečečnik and Jaka Sodnik (University of Ljubljana)