Hyper-V2X: Hypernetworks for Estimating Epistemic and Aleatoric Uncertainty in Cooperative Bird’s-Eye-View Semantic Segmentation

IEEE IV 2026 Oral

        

This paper introduces Hyper-V2X, a hypernetwork-based framework for estimating both epistemic and aleatoric uncertainties in V2X-based perception. Specifically, we propose

1. Bayesian hypernetwork formulation for cooperative perception.
2. V2X context embedding: that conditions a Bayesian hypernetwork on fused multi-agent features.
3. Partial weight generation for Bayesian hypernetworks; that avoids generating the full set of model parameters, enabling efficient and scalable uncertainty estimation.

Overview

Hyper-V2X is a cooperative perception model for autonomous driving that performs Bird’s-Eye-View (BEV) semantic segmentation with uncertainty estimation under communication constraints.

It leverages a Bayesian hypernetwork conditioned on fused multi-agent BEV features to generate stochastic decoder weights, enabling:

• Epistemic uncertainty estimation (model uncertainty)
• Aleatoric uncertainty estimation (data uncertainty)
• Robust BEV semantic segmentation in V2X settings
• Performance under limited communication bandwidth / compression constraints

Training Data

Hyper-V2X is trained on:

• OPV2V cooperative perception dataset

Dataset reference:

• https://huggingface.co/datasets/gqk/opv2v

Quick Start

For full installation instructions please refer to the Hyper-V2X GitHub repository.

Once the dependencies are installed, you can use the load the checkpoints from Hugging Face.

from torch.utils.data import DataLoader
import opencood.hypes_yaml.yaml_utils as yaml_utils
from opencood.tools import train_utils
from opencood.data_utils.datasets import build_dataset
from opencood.utils.seg_utils import (
    cal_iou_training,
    cal_ece_brier_score,
    cal_nll_brier_score
)

dataset = build_dataset(hypes, visualize=False, train=False)

    loader = DataLoader(
        dataset,
        batch_size=1,
        shuffle=False,
        num_workers=args.num_workers,
        collate_fn=dataset.collate_batch,
        pin_memory=False,
        drop_last=False
    )

    print("Loading model...")
    model = train_utils.create_model(hypes)

    device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
    model.to(device)
    print(model)

    _, model = train_utils.load_saved_model(args.model_dir, model)
    model.eval()

    # ---------------- RICH PROGRESS ----------------
    with torch.no_grad():
        with Live(refresh_per_second=4, console=console) as live:
            with Progress(
                TextColumn("Inference"),
                BarColumn(),
                TextColumn("{task.completed}/{task.total}"),
                TimeElapsedColumn(),
                TimeRemainingColumn(),
            ) as progress:

                task = progress.add_task("run", total=total)

                for i, batch_data in enumerate(loader):

                    batch_data = train_utils.to_device(batch_data, device)

                    model_out = model(batch_data['ego'])
                    post_output = dataset.post_process(batch_data['ego'], model_out)

                    # Segmentation Metric
                    iou_d, iou_s = cal_iou_training(batch_data, post_output)

                    ##### Uncertainty Metrics #########
                    ece, ece_eqp, _ = cal_ece_brier_score(batch_data, post_output)
                    nll, brier = cal_nll_brier_score(batch_data, post_output)

  

Citation

@article{jagtap2026hyper,
  title={Hyper-V2X: Hypernetworks for Estimating Epistemic and Aleatoric Uncertainty in Cooperative Bird's-Eye-View Semantic Segmentation},
  author={Jagtap, Abhishek Dinkar and Sadashivaiah, Sanath Tiptur and Festag, Andreas},
  journal={arXiv preprint arXiv:2605.21309},
  year={2026}
}