反因果域泛化：利用无标签数据
Anti-Causal Domain Generalization: Leveraging Unlabeled Data
针对目标环境分布偏移下的域泛化问题，在反因果设定中（结果变量导致观测协变量），环境扰动不影响结果变量，因此可通过正则化模型对这些扰动的敏感性来提升鲁棒性。估计扰动方向无需标签，从而能利用多环境中的无标签数据。提出两种方法，分别惩罚模型在环境间协变量均值和协方差的变化，并证明其在特定环境类下具有最坏情况最优性。在受控物理系统和生理信号数据集上验证了方法的有效性。
AuthorsSorawit Saengkyongam†, Juan L. Gamella, Andrew C. Miller†, Jonas Peters‡, Nicolai Meinshausen‡, Christina Heinze-Deml†
The problem of domain generalization concerns learning predictive models that are robust to distribution shifts when deployed in new, previously unseen environments. Existing methods typically require labeled data from multiple training environments, limiting their applicability when labeled data are scarce. In this work, we study domain generalization in an anti-causal setting, where the outcome causes the observed covariates. Under this structure, environment perturbations that affect the covariates do not propagate to the outcome, which motivates regularizing the model’s sensitivity to these perturbations. Crucially, estimating these perturbation directions does not require labels, enabling us to leverage unlabeled data from multiple environments. We propose two methods that penalize the model’s sensitivity to variations in the mean and covariance of the covariates across environments, respectively, and prove that these methods have worst-case optimality guarantees under certain classes of environments. Finally, we demonstrate the empirical performance of our approach on a controlled physical system and a physiological signal dataset.
- † Apple
- ‡ ETH Zürich
Related readings and updates.
Considerations for Distribution Shift Robustness in Health
This paper was accepted at the workshop “Trustworthy Machine Learning for Healthcare Workshop” at the conference ICLR 2023.
When analyzing robustness of predictive models under distribution shift, many works focus on tackling generalization in the presence of spurious correlations. In this case, one typically makes use of covariates or environment indicators to enforce independencies in learned models to guarantee…
More Speaking or More Speakers?
Self-training (ST) and self-supervised learning (SSL) methods have demonstrated strong improvements in automatic speech recognition (ASR). In spite of these advances, to the best of our knowledge, there is no analysis of how the composition of the labelled and unlabelled datasets used in these methods affects the results. In this work we aim to analyse the effect of number of speakers in the training data on a recent SSL algorithm (wav2vec 2.0),…
Discover opportunities in Machine Learning.
来源：Apple Machine Learning Research · machinelearning.apple.com