IDDA vs BDD100K

EXPERIMENT 2
From IDDA to BDD100K

It tests how well the networks trained with images taken from the synthetic world adapt to the real one and how the different distribution of data in IDDA affect the performance in the real target domain. To do so, we repeated the experiment two times considering as source domain two different IDDA distributions, one more similar and close to the real dataset (best case), the other much more different and faraway (worst case).

Source Scenario:

Best case scenario

  • Environment: Town 01, 02, 03, 04, 05, 06
  • Weather and illumination condition: Clear Noon
  • Viewpoint: Audi TT, Ford Mustang
  • Source train set size: 29952
Source Best Example Source Best Example Source Example Source Example Source Example Source Example
Click to see some sample taken from the best source scenario.

Worst case scenario

  • Environment: Town 07
  • Weather and illumination condition: Hard Rain Noon
  • Viewpoint: Jeep Wrangler(with hood), Bus(without hood)
  • Source train set size: 40128
Source Worst Example Source Worst Example Source Worst Example Source Worst Example Source Worst Example Source Worst Example
Click to see some sample taken from the worst source scenario.

Target/Test Scenario:

  • Dataset: BDD100K
  • Environment: 2 cities, New York and San Francisco
  • Weather and illumination condition: various
  • Viewpoints: different drivers
  • Target train set size (only with domain adaptation): 3333
  • Test size: 745
Target Example Target Example Target Example Target Example Target Example Target Example
Click to see some sample taken from the target/test scenario.

Results

Best Case Scenario

Experiment Distance Measurements Performance Evaluation
Euclidean
distance		 Cosine
distance		 Bhattacharaya
distance		 Network Code Available mIoU (%)
Source:
IDDA Best Case

Target:
BDD100K		 7,6177 1,6209 0,0502 without
domain
adaptation		 DeepLab V2 [1] (soon) 24,18
with
domain
adaptation		 DADA [2] (soon) 29,58
ADVENT [3] (soon) 33,18
CLAN [4] (soon) 33,47
DISE [5] (soon) 40,09

Worst Case Scenario

Experiment Distance Measurements Performance Evaluation
Euclidean
distance		 Cosine
distance		 Bhattacharaya
distance		 Network Code Available mIoU (%)
Source:
IDDA Worst Case

Target:
BDD100K		 7,7618 1,5526 0,0381 without
domain
adaptation		 DeepLab V2 [1] (soon) 17,48
with
domain
adaptation		 DADA [2] (soon) 23,45
ADVENT [3] (soon) 27,04
CLAN [4] (soon) 30,70
DISE [5] (soon) 31,37

Updated: