Accepted at IROS 2026

Chat2Scenic: An Iterative RAG-Based Framework for Scenario Generation in Autonomous Driving

The first iterative retrieval-augmented framework that translates natural-language regulatory descriptions into executable Scenic DSL scenarios for CARLA — driven by an interactive chatbot and grounded in regulatory knowledge.

Watch Demo Code Dataset Benchmark Paper (arXiv)
Chat2Scenic framework: Interactive Module, RAG Module, and Generation Module producing runnable CARLA scenarios.
The Chat2Scenic framework. An interactive chatbot interprets a natural-language query, a dual RAG retriever grounds generation in Scenic code and regulations, and a component-wise generator emits a runnable CARLA scenario. Click the figure to enlarge.

Yuan Gao1,*, Wenting Miao1,*, Mattia Piccinini1, Haoyu Wang1, Qunying Song2, Johannes Betz1

* Equal contribution.   1 Professorship of Autonomous Vehicle Systems, TUM School of Engineering and Design, Technical University of Munich, 85748 Garching, Germany; Munich Institute of Robotics and Machine Intelligence (MIRMI).   2 University College London, London, United Kingdom.

Technical University of Munich Autonomous Vehicle Systems Lab, TUM Munich Institute of Robotics and Machine Intelligence (MIRMI) University College London
1NL Description
2Interactive Interpreter
RRAG Dual Retriever
3Global Configuration
CComponent-wise Generation
4Scenic DSL
5CARLA Simulation

An interactive interpreter refines the query, a dual RAG retriever grounds generation in Scenic code and regulations, and a component-wise generator (Spatial Relation → Ego → Object → Restriction) emits an executable Scenic script for CARLA.

What's New

Key Contributions

Overview

Abstract

Validating autonomous driving systems requires diverse, regulation-compliant test scenarios. In simulation-based testing, scenarios are defined as executable DSL scripts that parametrically specify road topology, actor behaviors, and environmental conditions for simulators such as CARLA. Yet automatically generating such scripts from complex regulatory descriptions remains an open challenge, and existing approaches face fundamental trade-offs: retrieval-assemble methods achieve reasonable compilation rates but lack scalability, while retrieval-based full-script generation suffers from low compilation success. We present Chat2Scenic, the first iterative retrieval-augmented framework to automatically translate natural-language regulatory descriptions into executable Scenic DSL scripts. Chat2Scenic provides a chatbot interface for interactive scenario refinement and integrates RAG to ground scenario generation in regulatory knowledge and DSL syntax. We further propose an open benchmark comprising 123 scenarios drawn from NHTSA, United Nations Vehicle Regulations, and the CARLA Leaderboard. Chat2Scenic achieves a Compilation Success Rate of 76.42% and a Framework Accuracy of 58.17%, outperforming retrieval-assemble (30% CSR, 11.03% FA) and retrieval full-script generation (16.26% CSR, 10.08% FA). We release our open-source code to facilitate future developments.

123
benchmark scenarios
6
scenario source sets
(CARLA · NHTSA · UN)
76.42%
compilation success rate
58.17%
framework accuracy
Proprietary Gemini-3-Flash Gemini-3-Pro DeepSeek-V3.2 Qwen-Plus Qwen3-Coder-Plus
Open-source (Ollama) Qwen3:30B Qwen3-Coder:30B Gemma3:27B GPT-OSS:20B Mistral-Small3.2:24B
Live Walkthrough

System Demonstrations

Chat2Scenic ships two interactive tools — a conversational scenario generator and a dedicated human-evaluation interface — enabling end-to-end scenario creation and verification.

Demo 1 — Chatbot & Scenic Code Generation

Interactive Module

The Chat2Scenic chatbot lets users describe any driving scenario in plain natural language. The system iteratively retrieves relevant DSL knowledge via RAG, applies chain-of-thought prompting, and generates a component-wise executable Scenic DSL script ready for CARLA simulation. Users can refine the scenario through follow-up messages, inspect the generated code, and re-run the simulation — all within a single chat session.

Demo 2 — Human Evaluation GUI

Scenario Alignment Annotation

To assess scenario alignment, we built a dedicated annotation tool that presents evaluators with the original benchmark description alongside the CARLA simulation video (BEV, FPV, TPV views). Annotators rate five alignment layers — Road Layout (RL), Traffic Infrastructure (TI), Temporal Modifications (TM), Dynamic Objects (DO), and Environment (EN) — each on a binary pass/fail basis. The aggregated scores feed directly into the Scenario Quality (SQ) and Framework Accuracy (FA) metrics reported in our results.

Generated Scenarios

Qualitative Examples

Generated scenarios rendered in CARLA across benchmark categories. Each scenario shows all three camera views — BEV (Bird's-Eye), FPV (First-Person), TPV (Third-Person) — with the original benchmark description.

Evaluation

Quantitative Results

All experiments use CARLA 0.9.15 + Scenic 3.1.0 on a Dell Alienware R15 (Intel i7-13700KF, NVIDIA RTX 4090, 128 GB RAM). The best configuration C11 (CP + CoT + ICL + CodeICL) with Gemini-3-Flash significantly outperforms all baselines.

76.42%
Compilation Success Rate
76.12%
Scenario Quality
58.17%
Framework Accuracy

Ablation Study — Gemini-3-Flash, C1–C12

Click any numeric column header to sort. ↑ = higher is better, ↓ = lower is better.

ID ⇅ CP CoT ICL RAG-ICL CSR (%) ↑ ⇅ RT (s) ↓ ⇅ Tokens ↓ ⇅ Scenario Alignment ↑ FA ↑ ⇅
CodeICLDocICL RL ↑ ⇅ TI ↑ ⇅ TM ↑ ⇅ DO ↑ ⇅ EN ↑ ⇅ SQ ↑ ⇅

*C1 is zero-shot baseline (Gemini-3-Flash). CP: Contextual Prompting, CoT: Chain-of-Thought, ICL: In-Context Learning, RAG-ICL: Retrieval-Augmented ICL (Code/Doc). CSR: Compilation Success Rate, RT: Response Time, Tokens: token usage. Alignment layers — RL: Road, TI: Traffic Infrastructure, TM: Temporal Modifications, DO: Dynamic Objects, EN: Environment. SQ: Scenario Quality, FA: Framework Accuracy.

The prompting techniques

CP Contextual Prompting

Injects domain-specific knowledge directly into the prompt — syntax constraints, hierarchical type systems (e.g., NetworkElement → LinearElement → {Road, Lane}), and available operators.

CoT Chain-of-Thought

Structures generation into explicit reasoning steps per component (Understand → Examine → Select → Define → Instantiate → Validate), completing each step incrementally.

ICL In-Context Learning

Contrastive few-shot examples: positive examples show correct syntax patterns while negative examples present error-correction pairs, anchoring the model to concrete instances.

RAG-ICL Retrieval-Augmented ICL

Dynamic retrieval on top of ICL: CodeICL retrieves the top-3 similar code snippets per query, while DocICL retrieves relevant documentation chunks — always tailored to the component being generated.

Comparison across models & SOTA methods

Click any numeric column header to sort. ↑ = higher is better, ↓ = lower is better.

Model Variant ⇅ CSR (%) ↑ ⇅ RT (s) ↓ ⇅ Tokens ↓ ⇅ Scenario Alignment ↑ FA ↑ ⇅
RL ↑ ⇅ TI ↑ ⇅ TM ↑ ⇅ DO ↑ ⇅ EN ↑ ⇅ SQ ↑ ⇅

CSR: Compilation Success Rate, RT: Response Time. Alignment layers — RL: Road, TI: Traffic Infrastructure, TM: Temporal Modifications, DO: Dynamic Objects, EN: Environment. SQ: Scenario Quality, FA: Framework Accuracy (CSR × SQ). Highlighted rows: Chat2Scenic and SOTA baselines.

Dataset

Benchmark

The first regulation-grounded benchmark for autonomous-driving scenario generation from natural language, comprising 123 scenario specifications drawn from real-world regulatory sources. Browse all scenarios by source below.

123
Total Scenarios
24
CARLA Leaderboard
16
NHTSA Crash
31
NHTSA PreCrash
4
UN R152
12
UN R157
36
UN R171
Cite

BibTeX

@inproceedings{chat2scenic2026,
  title     = {Chat2Scenic: An Iterative RAG-Based Framework for
               Scenario Generation in Autonomous Driving},
  author    = {Gao, Yuan and Miao, Wenting and Piccinini, Mattia and
               Wang, Haoyu and Song, Qunying and Betz, Johannes},
  booktitle = {Proceedings of the IEEE/RSJ International Conference on
               Intelligent Robots and Systems (IROS)},
  year      = {2026},
  eprint    = {2607.14387},
  archivePrefix = {arXiv},
  primaryClass  = {cs.RO}
}