Display & Audio Routing

Advanced10 min read

Assumes: 6.1 QNX, 6.2 AAOS

TL;DR

Multiple OSs share physical displays and speakers on the same SoC. QNX renders the instrument cluster (safety-critical, always-on); AAOS renders the IVI center stack (infotainment). VirtIO-GPU lets the AAOS guest submit frames to a virtual GPU backed by QNX/hypervisor. Audio is harder than video: nav, media, phone, and chime sources compete with priority-based arbitration. Audio HAL handles stream I/O; Audio Control HAL handles routing, volume groups, and ducking. When cluster and IVI both glitch, suspect GPU contention or display pipeline config first.

Display and audio are where users feel the multi-OS architecture. Two operating systems, one windshield’s worth of pixels, one cabin full of speakers. Somehow it has to look seamless and never miss a chime. Simple question: “who controls the screen?” Answer: it’s complicated.

Multiple OSs, Multiple Displays

Modern cockpits have 2–4 physical displays driven from one SoC:

Display	Typical Owner	Content
Instrument cluster (behind steering wheel)	QNX	Speed, RPM, telltales, ADAS icons, trip computer
Center stack / IVI	AAOS	Navigation, media, climate UI, apps
Passenger display	AAOS (second display)	Entertainment, co-pilot nav
HUD (if present)	QNX or dedicated renderer	Speed, nav arrows, ADAS alerts

The ownership split is deliberate: safety-adjacent pixels live on QNX. If AAOS reboots during a drive, the cluster keeps showing speed and warnings. Cluster is QNX; IVI is AAOS.

flowchart TB
    subgraph Displays["Physical Displays"]
        CLUSTER["Instrument Cluster<br/>(1920×720 typical)"]
        IVI["Center Stack IVI<br/>(1920×1080 typical)"]
        PASS["Passenger Display"]
    end

    subgraph QNX["QNX VM"]
        QCOMP["Cluster Compositor"]
        QGL["OpenGL / Screen Graphics"]
    end

    subgraph AAOS["AAOS Guest VM"]
        SURF["SurfaceFlinger"]
        VGPU["VirtIO-GPU Driver"]
        APPS["Apps / System UI"]
        APPS --> SURF --> VGPU
    end

    subgraph HW["Display Hardware"]
        DPU["Display Processing Unit (DPU)"]
        PHY["DSI / eDP PHY"]
    end

    QCOMP --> QGL --> DPU
    VGPU -->|"VirtIO-GPU"| DPU
    DPU --> PHY
    PHY --> Displays

VirtIO-GPU: Guest Rendering

AAOS doesn’t drive the display hardware directly. It renders through VirtIO-GPU, a paravirtualized GPU device:

Concept	What Happens
Guest (AAOS)	Apps render → SurfaceFlinger → HWC/HAL → OpenGL/Vulkan → VirtIO-GPU driver
VirtIO protocol	GPU commands and frame buffers sent across VM boundary
Host (QNX/QHEE)	VirtIO backend maps guest frames to hardware display layers
DPU compositing	Hardware blends QNX cluster layer + AAOS IVI layer → physical output

Benefits:

• AAOS gets full GPU acceleration without direct hardware access
• QNX controls which layer goes to which display
• Hypervisor can limit GPU memory and schedule GPU time between VMs

Tradeoffs:

• GPU contention: cluster and IVI share the same physical GPU
• Extra latency: one VM boundary crossing for every frame submission
• Debug complexity: rendering bugs could be guest driver, VirtIO backend, or DPU config

Mental Model

Think of the DPU as a stack of transparent sheets (hardware layers). QNX paints the bottom sheet (cluster). AAOS paints a sheet above it (IVI) via VirtIO-GPU — like slipping a drawing through a slot in the desk to the shared light table. The light table (DPU) blends all sheets and projects to the screen. Neither OS sees the other’s pixels directly — they see their assigned layer.

VirtIO-GPU = one physical display, many virtual screens. DPU layers = Photoshop layers for the dashboard.

Display Routing: Who Gets Which Screen

Display assignment is configured at platform integration time, not runtime-discoverable:

Config Element	Set By	Example
Display IDs	BSP / device tree	Display 0 = cluster, Display 1 = IVI
Layer assignment	QNX display manager	QNX compositor → layer 0, VirtIO → layer 1
Resolution / timing	Panel specs	Cluster 1920×720@60, IVI 1920×1080@60
Multi-display AAOS	config.xml / overlay	Passenger display as display 2

Common Gotcha

Beginner thinks a display bug is one problem with one fix. Reality: “IVI works but cluster is black” and “cluster works but IVI is black” are different teams and different pipelines. Cluster black → QNX compositor, QNX display driver. IVI black → VirtIO-GPU, AAOS SurfaceFlinger, guest display config. Don’t assume one fix solves both.

Think About It

The driver says ‘the navigation voice is too quiet during phone calls.’ What’s the audio routing concept at play?

Reveal Answer

Audio priority arbitration and ducking: phone calls sit above navigation in the priority stack, so Audio Control HAL temporarily lowers nav stream gain (typically to ~30%) rather than muting it entirely. AudioFlinger routes nav PCM to bus1_nav_out; when phone audio focus is active, the Audio Control HAL ducking policy reduces that bus gain. Inspect active focus holders with adb shell dumpsys audio | grep -i focus and ducking rules via adb shell dumpsys car_service –audio.

Audio Routing: Multiple Sources, One Cabin

Audio is more contentious than video because speakers are shared. You can’t give nav and Spotify separate physical speakers (usually). Sources compete:

Source	Priority (typical)	Behavior When Higher Priority Speaks
Chime / warning	Highest	Mutes or ducks everything: collision alert, seatbelt
Phone call	High	Ducks media and nav
Navigation guidance	Medium-high	Ducks media, may duck with phone on hold
Voice assistant	Medium	Ducks media during response
Media (Spotify, radio)	Low	Ducked by everything above
System UI sounds	Low-medium	Brief, usually mixes or ducks media

Ducking means temporarily lowering volume of lower-priority streams (not muting entirely; nav might drop media from 100% to 30%). Chimes and calls win over everything else.

Audio HAL + Audio Control HAL

Android automotive splits audio into two HAL layers:

HAL	AIDL/HAL Name	Responsibility
Audio HAL	android.hardware.audio	Stream I/O: open output stream, write PCM buffers, capture from mic
Audio Control HAL	android.hardware.automotive.audiocontrol	Routing, volume groups, ducking policy, bus assignment

Audio HAL (Stream I/O)

Handles the mechanics:

• Open/close output streams per audio usage (media, navigation, voice, alarm)
• Write PCM data to the correct hardware audio bus (bus0_media, bus1_nav, etc.)
• Microphone capture for voice assistant and BT phone

Audio Control HAL (Routing & Policy)

Handles the intelligence:

• Map Android audio usages to physical output buses
• Define volume groups (media volume knob vs nav volume vs phone)
• Implement ducking rules: when nav speaks, tell Audio HAL to reduce media gain
• Coordinate with Car Service for occupant zone routing (driver vs passenger headphones)

flowchart TB
    subgraph AAOS["AAOS Audio Stack"]
        APP["Apps · Nav · Phone · Media"]
        AFW["AudioFlinger"]
        AH["Audio HAL"]
        ACH["Audio Control HAL"]
        APP --> AFW --> AH
        ACH -.->|"routing · ducking · volume"| AFW
        ACH -.->|"bus mapping"| AH
    end

    subgraph QNX["QNX Audio Path"]
        MIX["Audio Mixer / DSP"]
        AMP["Amplifier · Speakers"]
    end

    subgraph Sources["Priority Sources"]
        CHIME["Chimes / Warnings (QNX)"]
        NAV["Navigation (AAOS)"]
        MEDIA["Media (AAOS)"]
        PHONE["Phone (AAOS / BT)"]
    end

    AH -->|"PCM streams"| MIX
    CHIME -->|"Highest priority"| MIX
    MIX --> AMP

Note: safety chimes often originate on QNX. They bypass AAOS entirely and inject directly into the mixer. If AAOS crashes, you still hear the seatbelt chime.

Audio Bus Architecture (Automotive)

Automotive Audio HAL uses bus-based routing instead of phone-style deep buffer paths. Each logical bus maps to a physical output:

Android Bus	Typical Physical Output
bus0_media_out	Cabin speakers (media)
bus1_nav_out	Cabin speakers (nav; same physical, mixed)
bus2_phone_out	Cabin speakers or BT HFP
bus3_alarm_out	Chimes / warnings

The Audio Control HAL’s setBalanceTowardRight() and ducking APIs operate on these buses. Misconfigured bus mapping = “nav voice comes from wrong speaker.”

Display & Audio Routing Diagram

flowchart LR
    subgraph Video["Display Path"]
        QNX_D["QNX Cluster Renderer"]
        AAOS_D["AAOS SurfaceFlinger"]
        VGPU["VirtIO-GPU"]
        DPU["DPU Compositor"]
        QNX_D --> DPU
        AAOS_D --> VGPU --> DPU
        DPU --> PANELS["Cluster + IVI Panels"]
    end

    subgraph Audio["Audio Path"]
        QNX_A["QNX Chimes"]
        AAOS_A["AAOS AudioFlinger"]
        ACH["Audio Control HAL"]
        MIXER["Hardware Mixer / DSP"]
        SPK["Speakers"]
        QNX_A -->|"Priority 1"| MIXER
        AAOS_A --> ACH --> MIXER
        MIXER --> SPK
    end

Where You'll See This

Common display/audio tickets and first checks:

Ticket	First Look
Cluster flicker	QNX compositor logs, DPU underrun counters, GPU load
IVI black screen, cluster OK	adb shell dumpsys SurfaceFlinger, VirtIO-GPU dmesg, guest display config
Both displays glitch	GPU thermal throttling, shared memory pressure, hypervisor GPU scheduling
Nav voice silent	Audio Control HAL routing, bus1_nav_out mapping, nav app audio focus
Media doesn’t duck for nav	Ducking policy in Audio Control HAL, CarAudioService config
Chime too quiet / missing	QNX chime path (not AAOS), amplifier gain, mixer priority
Phone audio from wrong speaker	BT HFP routing, occupant zone config, bus2 mapping

Debug commands:

# AAOS display state
adb shell dumpsys SurfaceFlinger
adb shell dumpsys display

# AAOS audio routing
adb shell dumpsys media.audio_policy
adb shell dumpsys car_service --audio

# Active audio focus holders
adb shell dumpsys audio | grep -i focus

GPU contention between cluster and IVI often shows up as both stuttering simultaneously. That’s your clue it’s shared hardware, not one OS.

Connect the Dots

Module 6.1 (QNX) owns cluster rendering and chime injection. Module 6.2 (AAOS) owns IVI and media apps. Module 6.5 (Power Management) explains display/audio behavior during suspend: cluster may stay on while IVI sleeps. Module 0.1 (SoC Anatomy) introduced the GPU and DPU blocks these pipelines use.

Bottom line

1. Cluster = QNX, IVI = AAOS: safety pixels vs infotainment pixels.
2. VirtIO-GPU lets AAOS render without direct hardware access.
3. Audio uses priority arbitration: chimes > phone > nav > media.
4. Audio HAL = stream I/O; Audio Control HAL = routing, ducking, volume groups.
5. Safety chimes often bypass AAOS, injected from QNX directly.

Check Your Understanding

1. How does AAOS render graphics to the center stack display on a virtualized cockpit platform?

Applied

A. AAOS drives the display hardware directly via a bare-metal Adreno GPU driverB. AAOS renders through VirtIO-GPU; QHEE/QNX backend maps frames to DPU layerC. QNX copies Android framebuffer pixel data to the display panel every frameD. All rendering is done by QNX compositors; AAOS sends XML layout descriptions

2. Which audio source typically has the highest priority in automotive audio arbitration?

Recall

A. Navigation turn-by-turn guidance prompts with voice and tone alertsB. Safety chimes and collision warnings from the body or ADAS controllerC. Media playback audio streams such as Spotify and FM radio tuner outputD. Voice assistant responses from the Google Assistant or Alexa service

3. What is the primary role of the Audio Control HAL (separate from Audio HAL)?

Applied

A. Rendering the instrument cluster audio visualizer animation and frequency barsB. Managing audio routing, volume groups, ducking policy, and bus assignmentC. Encoding Bluetooth A2DP audio streams for wireless speaker connectionsD. Writing raw PCM audio buffers directly to the hardware output stream device

4. AAOS reboots mid-drive but the instrument cluster keeps showing speed and telltales. Which ownership model explains this?

Recall

A. Cluster and IVI share one Android process so cluster survives app restartB. AMSS modem renders the cluster display remotely via an LTE data connectionC. Cluster is QNX (safety domain); IVI is AAOS — independent display pipelinesD. The cluster display has its own separate AAOS instance running in a VM

5. Ticket: 'IVI black screen, cluster works fine.' Where do you investigate first?

Applied

A. QNX cluster compositor and the QNX display resource manager driverB. AMSS modem RF calibration tables for the cellular display data channelC. AAOS SurfaceFlinger, VirtIO-GPU guest driver, and guest display configD. CAN gateway signal database and DBC file for display property mapping

6. After 30 minutes of navigation, both cluster and IVI stutter simultaneously. GPU frequency is maxed and DDR bandwidth counters show saturation. Root cause?

Synthesis

Connects to: Module 0 — SoC Anatomy

A. Shared Adreno GPU and DDR bandwidth contention between QNX and AAOS compositorsB. VHAL property subscription rate too high causing Car Service thread starvationC. Android Auto phone projection conflict competing for the IVI display outputD. Separate CAN bus overload causing frame-drop signals affecting both displays

Further Reading

• Android Graphics Architecture — How SurfaceFlinger, display compositors, and hardware layers work in AAOS — the guest-side rendering path before VirtIO-GPU.
• QNX Screen Graphics Subsystem — QNX’s compositor and display framework for cluster rendering and DPU layer management on the host side.
• Android Automotive Audio — Automotive audio architecture including bus-based routing, volume groups, and ducking policy.
• Audio Control HAL (AIDL) — Official HAL specification for routing, ducking, and occupant-zone audio policy separate from stream I/O.
• VirtIO GPU Device Specification — OASIS VirtIO-GPU protocol reference explaining how AAOS guest frames cross the VM boundary to QNX/hypervisor backends.
• Android Multi-Display Support — How AAOS assigns displays (center stack, passenger) and how display IDs map to physical panels in multi-screen cockpits.
• Qualcomm Display Pipeline Overview — Qualcomm silicon context for DPU/GPU compositing shared between QNX cluster and AAOS IVI layers.