IDDA vs A2D2

EXPERIMENT 4
From IDDA to A2D2

It tests how well the networks trained with images taken from the synthetic world adapt to the real one and how the different distribution of data in IDDA affect the performance in the real target domain. To do so, we repeated the experiment two times considering as source domain two different IDDA distributions, one more similar and close to the real dataset (best case), the other much more different and faraway (worst case).

Source Scenario:

Best case scenario

  • Environment: Town 01, 02, 03, 04, 05, 06
  • Weather and illumination condition: Clear Noon
  • Viewpoint: Audi TT, Ford Mustang
  • Source train set size: 29952
Source Best Example Source Best Example Source Example Source Example Source Example Source Example
Click to see some sample taken from the best source scenario.

Worst case scenario

  • Environment: Town 07
  • Weather and illumination condition: Hard Rain Noon
  • Viewpoint: Jeep Wrangler(with hood), Bus(without hood)
  • Source train set size: 40128
Source Worst Example Source Worst Example Source Worst Example Source Worst Example Source Worst Example Source Worst Example
Click to see some sample taken from the worst source scenario.

Target/Test Scenario:

  • Dataset: Audi Driving Dataset - A2D2
  • Environment: three different German cities
  • Weather and illumination condition: various
  • Viewpoints: camera front center
  • Target train set size (only with domain adaptation): 18878
  • Test size: 3147
Target Example Target Example Target Example Target Example Target Example Target Example
Click to see some sample taken from the target/test scenario.

Results

Best Case Scenario

Experiment Distance Measurements Performance Evaluation
Euclidean
distance		 Cosine
distance		 Bhattacharaya
distance		 Network Code Available mIoU (%)
Source:
IDDA Best Case

Target:
A2D2*		 6,3874 1,0589 0,0447 without
domain
adaptation		 DeepLab V2 [1] (soon) 32,10
with
domain
adaptation		 DADA [2] (soon) 38,57
ADVENT [3] (soon) 42,56
CLAN [4] (soon) 44,31
DISE [5] (soon) 46,73

Worst Case Scenario

Experiment Distance Measurements Performance Evaluation
Euclidean
distance		 Cosine
distance		 Bhattacharaya
distance		 Network Code Available mIoU (%)
Source:
IDDA Worst Case

Target:
A2D2*		 7,0150 1,1849 0,0387 without
domain
adaptation		 DeepLab V2 [1] (soon) 29,80
with
domain
adaptation		 DADA [2] (soon) 36,18
ADVENT [3] (soon) 38,57
CLAN [4] (soon) 42,71
DISE [5] (soon) 45,49

*The number of classes considered during evaluation is 13 (w.r.t. to the other experiments where 16 classes in common between IDDA and the other real-world dataset are chosen).

Updated: