When Underwater Acoustic Recognition Fails Across Datasets: A Cross-Dataset Benchmark Revealing Zero-Transfer and Label Shift

Source abstract

Deep networks for underwater acoustic target recognition (UATR) are almost always trained and evaluated on a single dataset, leaving cross-dataset generalization unmeasured. We benchmark four ship-radiated-noise corpora—open-ocean Oceanship, freshwater-lake QiandaoEar22, an AIS-auto-labeled VTUAD reconstruction from public Ocean Networks Canada archives, and nearshore-harbor ShipsEar—under one preprocessing chain, frozen recording-level splits, and ImageNet-pretrained ResNet-18 classifiers. Direct transfer fails completely and systematically across all twelve cross-library directions: accuracies of 0.0–47.9% sit far below majority-class chance levels of 55.6–88.9%, and even the same-water Oceanship→VTUAD pair collapses to 0.9%, localizing a substantial part of dataset bias to labeling provenance and scenario definition rather than the acoustic channel. Centroid-level analysis shows the cross-dataset gap is 5–17 times the within-dataset inter-class distance. Under Oceanship’s 4,278:1 imbalance, sqrt-smoothed class weights raise macro-F1 from 0.1302 to 0.1452, whereas naive inverse weighting collapses training. We release the mapping tables, frozen splits, and evaluation code as a reproducible cross-dataset UATR benchmark.