Cal-SFDA: Source-Free Domain-adaptive Semantic Segmentation with Differentiable Expected Calibration Error
Paper
Paper/Presentation Title | Cal-SFDA: Source-Free Domain-adaptive Semantic Segmentation with Differentiable Expected Calibration Error |
---|---|
Presentation Type | Paper |
Authors | Wang, Zixin, Luo, Yadan, Chen, Zhi, Wang, Sen and Huang, Zi |
Journal or Proceedings Title | Proceedings of the 31st ACM International Conference on Multimedia (MM '23) |
Journal Citation | pp. 1167-1178 |
Number of Pages | 12 |
Year | 2023 |
Publisher | Association for Computing Machinery (ACM) |
Place of Publication | United States |
ISBN | 9798400701085 |
Digital Object Identifier (DOI) | https://doi.org/10.1145/3581783.3611808 |
Web Address (URL) of Paper | https://dl.acm.org/doi/10.1145/3581783.3611808 |
Web Address (URL) of Conference Proceedings | https://dl.acm.org/doi/proceedings/10.1145/3581783 |
Conference/Event | 31st ACM International Conference on Multimedia (MM '23) |
Event Details | 31st ACM International Conference on Multimedia (MM '23) Parent ACM International Conference on Multimedia Delivery In person Event Date 29 Oct 0202 to end of 03 Nov 2023 Event Location Ottawa, Canada |
Abstract | The prevalence of domain adaptive semantic segmentation has prompted concerns regarding source domain data leakage, where private information from the source domain could inadvertently be exposed in the target domain. To circumvent the requirement for source data, source-free domain adaptation has emerged as a viable solution that leverages self-training methods to pseudo-label high-confidence regions and adapt the model to the target data. However, the confidence scores obtained are often highly biased due to overconfidence and class-imbalance issues, which render both model selection and optimization problematic. In this paper, we propose a novel calibration-guided source-free domain adaptive semantic segmentation (Cal-SFDA) framework. The core idea is to estimate the expected calibration error (ECE) from the segmentation predictions, serving as a strong indicator of the model's generalization capability to the unlabeled target domain. The estimated ECE scores, in turn, assist the model training and fair selection in both source training and target adaptation stages. During model pre-training on the source domain, we ensure the differentiability of the ECE objective by leveraging the LogSumExp trick and using ECE scores to select the best source checkpoints for adaptation. To enable ECE estimation on the target domain without requiring labels, we train a value net for ECE estimation and apply statistic warm-up on its BatchNorm layers for stability. The estimated ECE scores assist in determining the reliability of prediction and enable class-balanced pseudo-labeling by positively guiding the adaptation progress and inhibiting potential error accumulation. Extensive experiments on two widely-used synthetic-to-real transfer tasks show that the proposed approach surpasses previous state-of-the-art by up to 5.25% of mIoU with fair model selection criteria. |
Keywords | source-free domain adaptation; semantic segmentation; calibration |
Contains Sensitive Content | Does not contain sensitive content |
ANZSRC Field of Research 2020 | 4602. Artificial intelligence |
Public Notes | © 2023 Association for Computing Machinery. This is the author's version of the work. It is posted here for your personal use. Not for redistribution. The definitive Version of Record was published in MM '23: Proceedings of the 31st ACM International Conference on Multimedia, |
Byline Affiliations | University of Queensland |
https://research.usq.edu.au/item/zyx42/cal-sfda-source-free-domain-adaptive-semantic-segmentation-with-differentiable-expected-calibration-error
Download files
11
total views0
total downloads7
views this month0
downloads this month