MapAgent: An Industrial-Grade Agentic Framework for City-scale Lane-level Map Generation

Following prior works on lane-level map generation and updating (Xia et al., 2024, 2025), the general task is to convert a BEV observation into a standardized vectorized lane map. Concretely, given a BEV observation collected from vehicle-mounted sensors, a map generation or updating system aims to predict a structured vectorized map , where each map element is represented in a unified vectorized form with its geometry and attributes. In this paper, we formulate MapAgent as a constrained map refinement problem built on top of a frozen BEV vectorization backbone. Given a BEV observation and an initial draft map produced by the backbone, MapAgent iteratively edits into a specification-compliant map.

At refinement step , we treat as the environment state. Let denote the set of low confidence lane instances selected by the Quality Agent for refinement.

The Judge produces a lane-wise structured diagnosis for each ,

(1)

and we aggregate them into a step-level diagnosis

(2)

Conditioned on , the Planner is implemented as a rule-based module that generates a tool-grounded refinement plan:

(3)

Concretely, is an ordered sequence of actions

(4)

Each action follows a fixed schema (consistent with Worker tools), e.g., , and the plan is executed deterministically to update the map via

(5)

All edits must satisfy an immutable feasibility gate (geometric and topological validity, mapping specifications); edits that lead to are rejected by design (see App. A). Overall, MapAgent is driven by a learned Judge that produces structured diagnoses (trained via SFT and RL; see Sec. 2.3), and a rule-based Planner that generates minimal valid plans under using a closed action set defined by Worker tools.

As illustrated in Figure 2, MapAgent is an agent-based refinement framework on top of a frozen backbone. It builds upon city-scale BEV vectorization backbones to retain strong perception performance and scalability, while introducing an explicit agentic verification-and-editing loop that enforces lane-level specifications. The Quality Agent applies an early-acceptance fast track to tiles whose confidence score exceeds a fixed threshold , while only tiles with confidence below are forwarded to a bounded refinement loop. This selective routing focuses computation on difficult regions while preserving the throughput of the backbone. In practice, most tiles are directly accepted, and only a minority of hard tiles require iterative refinement. The separation between fast-track acceptance and iterative correction also simplifies system integration, as the backbone remains unchanged.

For each forwarded tile, MapAgent runs an iterative Judge–Planner–Worker workflow with re-validation. We explicitly bound the outer refinement budget by a small constant, three rounds in our system, to ensure predictable latency and prevent over-editing. This bounded design provides stable runtime behavior and avoids cascading modifications. At each round, the system evaluates the current map state before applying the next plan, ensuring that only validated edits are retained. If the refinement converges earlier, the loop terminates immediately. When the budget is exhausted without further admissible improvement, the system outputs the best validated map state.

As shown in Figure 3, the Judge Agent conducts geometry and topology verification, enforces specification compliance, and produces structured evidence for downstream refinement.

For each inspected lane , the Judge produces a structured, scalar-sized diagnostic entry

(6)

where is the predicted error category drawn from the predefined error set , is the associated confidence, and is a concise evidence summary. The taxonomy is derived from large-scale historical post-editing logs; newly observed anomalies are reviewed through a human-in-the-loop process, clustered, and used to update mapping specifications and subsequent Judge retraining data. This fixed schema is the sole interface exposed to the Planner: internal artifacts (e.g., chain-of-thought traces) are retained for training and debugging only and are not directly consumed by downstream modules. Applying the Judge to all yields lane-wise diagnoses , which are aggregated as and passed to the Planner at round .

To create supervised training instances, we sample individual lanes from the backbone draft map and construct model inputs by overlaying the target lane onto the BEV observation as a semi-transparent mask to focus attention on local evidence. Each instance is paired with a ground-truth error label and, optionally, an explanatory reference trace used for SFT. We denote a single supervised trajectory for lane as , where denotes the -th intermediate reasoning rationale, and the supervised corpus as . Note that the lane-masking strategy is a training-time input construction intended to focus attention on local evidence while preserving the global context through the unmasked portions of . Thus, the Judge is lane-centered but not lane-isolated: it still conditions on the full BEV observation and current map context, allowing it to reason about consistency with adjacent lanes.

The Judge Agent is implemented as an autoregressive multimodal policy that conditions on the BEV , the current map context , the selected lane , and previously generated tokens to produce a short reasoning sequence followed by the final structured prediction .

We first perform supervised fine-tuning on by maximizing the likelihood of the reference traces and final labels. To maintain stability and avoid catastrophic changes to the pretrained backbone, only lightweight adapter parameters are updated during SFT while the backbone weights remain frozen.

After SFT, we further apply GRPO to align the Judge with the downstream Planner–Worker objective. Given a lane-level input , the Judge policy generates an output consisting of a concise reasoning trace, a final error type, a confidence score, and an evidence summary. For each input , GRPO samples a group of candidate outputs from the old policy,

(7)

and computes a scalar reward for each candidate . Unlike PPO, GRPO does not require a separate value model; instead, it estimates the advantage by normalizing rewards within the sampled group:

(8)

This group-relative normalization reduces memory cost during VLM fine-tuning and naturally fits our setting, where multiple candidate diagnoses for the same lane can be directly compared.

We optimize the clipped surrogate

(9)

The GRPO objective is then defined as

(10)

where is the frozen SFT policy. The KL penalty prevents the updated Judge from drifting away from the structured output behavior learned during SFT.

For each candidate output, the reward is computed as

(11)

We compute the terms in an executability-first order. acts as a hard gate: if the response cannot be parsed as JSON, the reward computation stops with ; if the JSON misses required fields or the evidence mentions lane identifiers outside the valid metadata set, we also assign a executability penalty. A valid schema with no hallucinated lane identifier receives a small format reward . is when the predicted error_type matches the oracle label and otherwise. starts from zero and checks the reasoning trace: it subtracts if the trace does not contain exactly four sentences, subtracts if it mentions any lower-priority error type than the final prediction, and subtracts if required higher-priority exclusions are missing. If none of these rule violations occurs, . During GRPO training, we update only the LoRA adapter parameters and keep the pretrained VLM backbone frozen, preserving visual grounding while improving Judge reliability on hard cases under industrial map-production constraints.

We do not train the full Judge–Planner–Worker stack end-to-end. This is intentional: the Planner encodes immutable mapping specifications and the Workers are deterministic executors, so exposing them to unconstrained policy optimization would weaken safety guarantees. The learned component is therefore restricted to the Judge, while Planner/Worker behavior remains auditable and version-controlled.

The Planner Agent is a decision-making module that converts lane-level quality assessments produced by the Judge Agent into executable refinement plans under strict mapping constraints.

At refinement step , the Planner receives the structured Judge outputs for candidate lane lines and consumes the pair to generate the corrective plan . When required for consistency, the Planner may additionally query the historical map state to reason about past modifications. This query acts as an anti-oscillation filter and is triggered only when the same lane is edited in consecutive iterations; in production logs, this occurs in less than of refinement cases. No perceptual features beyond these structured inputs are available to the Planner.

The Planner outputs a structured refinement plan following a fixed schema. A plan consists of an ordered sequence of actions as defined in Eq. 4, where each action is defined by a tool type, a target lane identifier, and tool-specific parameters. If all lanes are judged as correct, the Planner outputs an empty plan .

All planning decisions are constrained by externally defined and immutable mapping rules that encode lane-level specifications and safety constraints. These rules prohibit the creation of new lane lines, cross–lane-group modifications, and large-scale structural changes. Every generated plan must satisfy both schema validity and rule compliance before execution; plans that fail validation are rejected and treated as empty plans.

The Planner maintains a lightweight memory of past refinement actions and their outcomes, which is used to filter redundant or conflicting decisions. If no valid plan can be constructed under the imposed constraints and current map state, the Planner outputs an empty plan and terminates refinement. This conservative fallback mechanism prevents cascading errors and reduces the risk of degrading the map when corrective actions are uncertain or inadmissible.

MapAgent operates under a bounded refinement budget and a closed, specification-verified tool set. The Worker tools are deterministic lane-line editors that support only local, auditable modifications. They cannot create new lane instances or perform cross-lane-group/non-local topology modifications. Deletion is allowed only for lanes diagnosed as redundant or spurious and is accepted only after passing the feasibility gate . Therefore, the Workers are designed to avoid unsafe global topology changes while enabling local, auditable correction.

The Worker Agent is responsible for carrying out the refinement plans generated by the Planner Agent. MapAgent exposes a fixed and limited set of Worker tools, each designed for a specific type of lane-level refinement: (i) a Category Worker for semantic label correction, (ii) a Delete Worker for removing redundant or spurious lane lines, (iii) a Smooth Worker for local geometric smoothing, and (iv) a Regenerate Worker for local geometry repair using a trained model (SAM3) (Carion et al., 2025). Among the available tools, the Regenerate Worker is the only one that leverages a learned model. It is designed for local geometry repair within the spatial support of an existing lane line, and its behavior is bounded by Planner-specified constraints, targeting local inconsistencies such as broken segments or misaligned geometry. Refinement actions are executed sequentially according to the order specified in the plan .

Dataset Construction. Following the DuLD dataset construction protocol in DuMapNet (Xia et al., 2024), we build large-scale lane-level vector maps from the Baidu Map Database by rendering a high-quality offline BEV image via multi-trajectory aggregation of camera–LiDAR fusion signals and assembling the vectorized ground truth from lane-related instances and attributes within each region (grouped by lane-group IDs, transformed into the local BEV coordinate system, and filtered with basic consistency operations such as removing invalid geometries and normalizing attributes). On top of this base dataset, we further curate a challenging hard subset using a backbone-agnostic difficulty criterion (e.g., high junction complexity and occlusion proxies derived from map topology / scene metadata), and use the same subset consistently across all backbones to ensure fairness. The training split contains 3,712 BEV images with 59,434 ground-truth lane instances; DuMapNet and GeMap produce 59,928 and 50,263 predicted lane instances, respectively. The test split contains 656 images with 10,254 ground-truth lanes; DuMapNet and GeMap produce 10,340 and 8,734 predicted lane instances, respectively.

Metrics. We measure lane-level correction quality by matching each predicted lane to the most appropriate ground-truth lane and then computing metrics on the resulting assignments. Accuracy, Precision, Recall, and F1 are defined on fully correct lanes: a prediction is counted as a true positive only if it can be matched to a ground-truth lane and is correct as a whole, i.e., it satisfies the matching criterion and has the correct lane category. Predictions without a valid match are treated as false positives, and unmatched ground-truth lanes are treated as false negatives. In contrast, BBox IoU and Mask IoU quantify geometric overlap, and Cls Acc measures category correctness, all computed only over matched lane pairs.

Backbones and Protocol. We evaluate MapAgent as a post-hoc refinement module on top of two representative BEV vectorization backbones, GeMap and DuMapNet. Unless otherwise noted, backbone predictors are frozen and MapAgent is applied without retraining. For the VLM-based Judge Agent, we compare different base models, including Qwen3-VL-Instruct (8B), Qwen3-VL-Thinking (8B), and InternVL-3.5-8B, under an identical prompting and inference protocol (same prompt template, decoding strategy, and refinement budget). All hyperparameters related to refinement (e.g., maximum number of tool calls and termination criteria) are fixed across experiments.

Hardware and Software. All experiments were run on a single server equipped with NVIDIA A800 80GB GPUs. The software stack is PyTorch 2.6.0 with CUDA 12.4.

VLM, SFT and GRPO. The Judge Agent module is based on the model Qwen3-VL-8B-Thinking (approximately 8B parameters).

The model is initialized from a publicly available pretrained checkpoint and further adapted using parameter-efficient fine-tuning. Specifically, we first apply supervised fine-tuning (SFT) with LoRA using a learning rate of , batch size of , LoRA rank , and epochs. On top of the SFT checkpoint, we perform a lightweight GRPO stage to better align Judge outputs with downstream map-refinement objectives. GRPO updates only the LoRA parameters with learning rate , rollout batch size , GRPO clip , KL coefficient , and epochs per update.

Runtime Measurements. After warm-up tiles, we measure runtime over randomly sampled validation tiles (including data I/O). The full MapAgent pipeline achieves a mean latency of  ms/tile, median  ms, p95  ms, and p99  s. Module-level averages are  ms per tile for the Judge Agent (p95  ms) and  ms/tile for the Worker (SAM + regenerate, p95  ms). Peak GPU memory usage is approximately  GB per A800. MapAgent is triggered on about of tiles in the test set.

Table 1 details the performance of VLM-based Judges on lane-quality inspection. SFT yields 58.23% for InternVL-3.5-8B and 70.16% for Qwen3-VL-8B, while subsequent GRPO alignment yields further targeted gains. For instance, GRPO improves the overall accuracy of Qwen3-VL-8B-Thinking from 83.55% to 86.01% and improves most class-wise precision/recall values. However, the Structure Error category shows a slight drop, suggesting a trade-off between overall decision accuracy and this minority/harder error type.

The comparison also shows that reasoning-oriented VLMs are more suitable for priority-based quality inspection. Qwen3-VL-8B-Thinking substantially outperforms the non-thinking Qwen3-VL-8B under SFT, indicating that explicit reasoning helps the Judge distinguish visually similar error types and follow the predefined short-circuit order. After GRPO, the model further improves on no_error, extra_lane_line, category_error, and geometry_error, suggesting that reward-based alignment makes the Judge outputs more consistent with executable downstream refinement.

Table 3 evaluates MapAgent as a frozen post-hoc refinement layer for GeMap and DuMapNet. MapAgent consistently improves lane-level correctness, scaling with Judge capability. On GeMap, Qwen3-VL-Thinking lifts Accuracy from 52.8 to 61.3 and F1 from 69.1 to 76.0. DuMapNet sees even larger gains (Accuracy 52.263.9, F1 68.678.0). Stable improvements across various models suggest MapAgent benefits systematically from stronger visual–language judgments rather than specific architectures.

Notably, gains are concentrated in Precision, Recall, and classification correctness, while geometric metrics (e.g., IoU) remain stable. This aligns with MapAgent’s role as a specification-aware editor: it corrects spurious lanes, category mismatches, and match failures caused by local geometry/category errors rather than aggressively altering lane geometry. The pronounced improvements on DuMapNet further indicate the refinement loop is particularly effective at rectifying errors in initial draft maps.

To better understand which parts of MapAgent drive the gains, we run compact ablations under the same unified evaluation protocol as Table 3. For readability, Table 2 reports the averaged results over GeMap and DuMapNet, and we only show the final map quality after post-hoc refinement. These ablations focus on two key factors: whether the Judge provides structured evidence beyond error labels, and how much benefit can be obtained from iterative correction.

We first examine what happens when the Judge is reduced to a pure error-type classifier. When we remove the explicit reasoning/evidence generation and let the Judge output the error type only (w/o Reason), the system still improves noticeably over the frozen base predictors, moving Accuracy from 52.5 to 58.4 and F1 from 68.9 to 73.7, with Cls Acc rising from 90.0 to 94.8. However, the gap to the full MapAgent remains clear: with the complete Judge–Planner–Worker loop, F1 reaches 77.0 and Cls Acc reaches 98.0. This difference suggests that, beyond recognizing what is wrong, the Judge’s structured reasoning is important for producing actionable, localized evidence that the Planner can reliably translate into safe, tool-grounded edits. In other words, reasoning improves not only interpretability, but also the executability of subsequent correction.

We then vary the maximum correction budget in the bounded retry loop. A single round already captures a large portion of the benefit (: 58.3 Accuracy and 73.6 F1), indicating many errors can be fixed with one pass of diagnosis and execution. Allowing a second iteration brings a further jump (: 60.3 Accuracy and 75.2 F1), and a third round continues to help, though with diminishing returns (: 62.6 Accuracy and 77.0 F1). This trend shows that iterative refinement is useful, but most easy-to-correct cases are resolved in the first few rounds. Across these settings, BBox/Mask IoU change only slightly (e.g., 70.4/35.0 at baseline to 71.8/36.0 at ), which aligns with MapAgent’s conservative design: most gains come from resolving discrete false positives/negatives and type mistakes rather than aggressively deforming lane geometry or altering topology.

Figure 4 shows several examples from the hard subset. As scenes become more challenging—e.g., with faint, missing, or occluded lane markings—both GeMap and DuMapNet often produce messy predictions, including spurious lanes, fragmented segments, and inconsistent topology. This failure mode is not specific to a particular backbone, but reflects a general limitation of feed-forward map prediction under weak or ambiguous visual evidence.

MapAgent effectively refines these noisy predictions into cleaner and more structured maps. The corrected results contain fewer spurious segments and exhibit more consistent global topology. In practice, the refinement focuses on obvious structural issues—removing hallucinated lanes, suppressing fragments, and restoring coherent layouts—while keeping geometry conservative. This qualitative trend matches the quantitative gains in lane-level correctness and demonstrates the method’s scalability.

High-definition map construction has evolved from multi-stage segmentation pipelines to end-to-end vectorized regression (Li et al., 2022). VectorMapNet (Liu et al., 2023) and MapTR (Liao et al., 2022) advanced direct polyline decoding with autoregressive and permutation-invariant transformers, respectively, while recent systems further improve geometric consistency and structured modeling, e.g., GeMap (Zhang et al., 2023) and HiMap (Zhou et al., 2024). Beyond purely visual cues, a growing line of work leverages map priors to stabilize matching and extend range, including Neural Map Prior (Xiong et al., 2023), P-MapNet (Jiang et al., 2024), and PriorMapNet (Wang et al., 2024), which incorporate SD maps or historical/outdated priors as additional conditioning signals. More recently, SDTagNet (Immel et al., 2025) exploits text-annotated SD maps to enhance far-range online HD map construction. In parallel, mixture-of-experts and interaction-based designs (e.g., MapExpert (Zhang et al., 2025) and InteractionMap (Wu et al., 2025)) improve long-tail element modeling via expert routing and structured temporal–spatial interactions. Concurrently, VLM-based efforts such as MapGPT (Zhang et al., 2024) and MAPLM (Cao et al., 2024) suggest that multimodal reasoning may help interpret maps and traffic scenes. However, these methods largely remain monolithic predictors that optimize geometric/semantic metrics and treat cartographic standards and traffic regulations as implicit supervision, leaving specification violations to manual post-editing. MapAgent addresses this gap by serving as a specification-aware refinement layer on top of strong BEV backbones.

LLM/VLM-based agents have rapidly advanced in tool use and self-correction. ReAct (Yao et al., 2022) interleaves reasoning and acting for interactive problem solving, Toolformer (Schick et al., 2023) learns to decide when/how to call tools, and Reflexion (Shinn et al., 2023) improves agents via feedback-driven memory. Recent work further strengthens intrinsic self-correction via multi-turn RL (e.g., SCoRe (Kumar et al., 2024)). Complementary lines study modular tool routing and grounded execution (e.g., MRKL (Karpas et al., 2022), SayCan (Ahn et al., 2022), ReWOO (Xu et al., 2023), Voyager (Wang et al., 2023)). Meanwhile, general-purpose multimodal foundations (e.g., GPT-4 (Achiam et al., 2023), Qwen-VL (Bai et al., 2023), InternVL (Chen et al., 2024), PaLM-E (Driess et al., 2023)) provide strong visual grounding and verification capabilities. While agentic workflows have been applied to driving planning (e.g., DriveLM (Sima et al., 2024)), they have not been adapted for the strict safety constraints of map production. MapAgent pioneers this paradigm in mapping by combining VLM-based specification verification with deterministic tool execution, ensuring industrial-grade compliance.