The focus is on the use of small language models (SLM) such as  Microsoft’s Phi-4-mini-instruct in Microsoft Foundry that is powered by NVIDIA DRIVE AGX accelerated compute. The solution allows vehicle functions such as the dashboard or individual vehicle profiles to be configured by voice command. At the same time, the system acts as robust local backup intelligence for cloud-based large language models (LLMs).

More intelligence, robustness and efficiency in the software-defined vehicle

Embedded small language models increase the functional intelligence level of modern vehicles and improve the responsiveness and availability of AI-powered functions. Since inference takes place directly in the vehicle, central functions remain available even when there is a limited or no internet connection. In addition, embedded SLMs enable a reduction in backend and infrastructure costs, as cloud-based LLMs can be supplemented or partially replaced depending on the use case. This helps OEMs economically scale software-defined vehicle functions.

“Our collaboration with Microsoft and NVIDIA showcases how small, efficient language models can transform in-vehicle experiences, delivering powerful functionality without the overhead of larger systems,” said Thomas Hülshorst, Group Vice President Intelligent Mobility and Software at FEV.

“By combining advanced AI frameworks with domain- and task-specific optimizations, FEV and Microsoft are shaping the future of intelligent, voice-driven interfaces that meet the high standards of automotive deployment,” added Boris Scholl, Vice President of Engineering at Microsoft.

Central fields of application for embedded GenAI

As part of the cooperation, FEV is investigating several fields of application with high series potential:

• Automated and autonomous driving (SAE Levels 3-5): Multimodal GenAI models improve the recognition of objects, traffic situations and driving paths, especially in complex urban environments and edge cases.
• Driver and passenger monitoring: Embedded GenAI increases performance in detecting fatigue, distraction, or unusual behavior, and increases the robustness of security-relevant functions through local availability – also as a backup to cloud-based systems.
• Personalized vehicle and HMI configuration: Vehicle functions and user interfaces can be intuitively adapted by voice command, for example for different driver profiles or usage scenarios – without dependence on external cloud infrastructure.

Multimodal System Architecture on NVIDIA Platforms

The underlying architecture is designed as a multimodal system and processes speech, text and visual information. To achieve high performance in the constrained environment of an embedded in-vehicle environment, FEV has used synthetically generated data curated with NVIDIA NeMo in the fine-tuning process to optimize the Phi-4-mini-instruct model. This is followed by integration and deployment of the resulting model running on NVIDIA DRIVE AGX enabling further improvement in model performance. The AI functions are operated as modular software services inside the vehicle.

Showcase: Personalized vehicle and HMI configuration

Building on these technologies, FEV has developed a dashboard configurator showcase that highlights the potential of intelligent in-vehicle interfaces. Using natural voice commands, a locally deployed small language model dynamically updates the dashboard—reducing reliance on continuous cloud connectivity while enabling fast, responsive performance.

Completed in just a few weeks, the showcase demonstrates strong real-time performance. In the future, FEV plans to gradually supplement or replace cloud-based AI functions via models that run locally in the vehicle.

FEV as a development partner for GenAI solutions

FEV is contributing its extensive expertise in the areas of software-defined vehicles, AI integration and additional development solutions. The company’s portfolio spans the entire value chain, from use case definition through architecture and system design to validation and industrialization. The results of the collaboration are currently being tested in demo vehicles and will serve as the basis for further customer projects and near-series applications later this year. In addition, FEV is focused on bidirectional communication between the driver/passenger and vehicle across a wide range of use-cases.